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A  special  form  for  modifica.tion  of  neuronal  response  properties  is  described 
in  which  the  change  in  the  synaptic  state  vector  is  parallel  to  the  vector 
of  afferent  activity.  This  process  is  termed  "parallel  modification"  and  its 
theoretical  and  experimental  implications  are  examined. 

A  theoretical  framework  has  been  devised  to  describe  the  complementary 
functions  of  generalization  and  discrimination  by  single  neurons.  This 
constitutes  a  basis  for  three  models  each  describing  processes  for  the  ^  ^ 
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development  of  maximum  selectivity  (discrimination)  and  minimum  selectivity 
(generalization)  by  neurons.  Strengthening  and  weakening  of  synapses  is 
expressed  as  a  product  of  the  presynaptlc  activity  and  a  nonlinear  modulatory 
function  of  two  postsynaptic  variables— namely  a  measure  of  the  spatially 
integrated  activity  of  the  cell  and  a  temportal  integration  (time-average)  of  j 
that  activity.  )  Some  theorems  are  given  for  low-dimensional  systems  and  a 
computer  simulation  results  from  more  complex  systems  are  discussed. 

^odel  neurons  that  achieve  high  selectivity  mimic  the  development  of  cat 
visual  cortex  neurons  in  a  wide  variety  of  rearing  condidltons.  A  role  for 
low-selectivity  neurons  is  proposed  in  which  they  provide  inhibitory  input  to 
neurons  of  the  opposite  type,  thereby  suppressing  the  common  component  of  a 
pattern  class  and  enhancing  their  selective  properties .\  Such  contrast-enhanc¬ 
ing  circuits  are  analyzed  and  supported  by  computer  simulation.  To  enable 
maximum  selectivity,  the  net  inhibition  to  a  cell  must become  strong  enough 
to  offset  whatever  excitation  is  produced  by  the  non^preferred  patterns. 

Ramifications  of  parallel  models  for  certain  experimental  paradigms  are 
analyzed,^  A  methodology  is  outlined  for  testing  synaptic  modification 
hypotheses  in  the  laboratory.  A  plastic  projection  from  one  neuronal  popula- 

-rfon  to  another  will  attain  stable  equilibrium  under  periodic  electrical  stim¬ 
ulation  of  constant  Intensity.  The  perturbative  effect  of  shifting  this 
intensity  level  can  yield  important  information  regarding  the  form  of  the 
modification  equation.  The  pathway  from  entorhinal  cortex  to  dentate  gyrus 
in  the  rat  is  an  ideal  candidate  for  such  experiments. 
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A  special  foru  for  modification  of  neuronal  response  properties 
is  described  in  which  the  change  in  the  synaptic  state  vector  is 
parallel  to  the .vest or _of  afferent  activity.  This  process  is  tensed 
parallel  nodification  and  its  theoretical  and  experimental 
implications  are  examined. 

A  theoretical  framework  has  been  devised  to  describe  the 
complementary  functions  of  generalisation  and  discrimination  by 
single  neurons.  This  constitutes  a  basis  for  three  models  each 
describing  processes  for  the  development  of  maximum  selectivity 
(discrimination)  and  minimum  selectivity  (generalisation)  by  neurons. 
Strengthening  and  weakening  of  synapses  is  expressed  as  a  product  of 
the  presynaptic  activity  and  a  nonlinear  modulatory  function  of  two 
postsynaptic  variables  --  namely  a  measure  of  the  spatially 
integrated  activity  of  the  cell  and  a  temporal  integration  (time- 
average)  of  that  activity.  Some  theorems  are  given  for  low¬ 
dimensional  systems  and  computer  simulation  results  from  more  complex 
systems  are  discussed. 

Model  neurons  that  achieve  high  selectivity  mimic  the 
development  of  cat  visual  cortex  neurons  in  a  vide  variety  of  rearing 
conditions.  A  role  for  low-selectivity  neurons  is  proposed  in  which 
they  provide  inhibitory  input  to  neurons  of  the  opposite  type, 
thereby  suppressing  the  common  component  of  a  pattern  class  and 
enhancing  their  selective  properties.  Such  contrast-enhancing 
circuits  are  analyzed  and  supported  by  computer  simulation.  To  enable 
maximum  selectivity,  the  net  inhibition  to  a  cell  must  become  strong 
enough  to  offset  whatever  excitation  is  produced  by  the  non¬ 
preferred  patterns. 

Ramifications  of  parallel  models  for  certain  experimental 
paradigms  are  analysed.  A  methodology  is  outlined  for  testing 
synaptic  modification  hypotheses  in  the  laboratory.  A  plastic 
projection  from  one  neuronal  population  to  another  will  attain  stable 
equilibrium  under  periodic  electrical  stimulation  of  constant 
intensity.  The  perturbative  effect  of  shifting  this  intensity  level 
can  yield  important  information  regarding  the  form  of  the 
modification  equation.  The  pathway  from  entorhinal  cortex  to  dentate 
gyrus  in  the  rat  is  an  ideal  candidate  for  such  experiments. 
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Introduction 


That  synaptic  modification  is  a  principal  component  of  neural 
plasticity  has  come  to  be  a  widely  held  opinion  among 
neuroscientists.  Theoreticians  have  launched  several  attempts*  with 
varying  degrees  of  success*  to  deduce  explicit  (synaptic)  principles 
underlying  learning  by  examining  the  se lforg&nizational  consequences 
of  various  microscopic  hypotheses  for  comparison  with  anatomical* 
physiological*  and  behavioral  data.  Such  efforts  have  been. directed 
toward  several  brain  structures  and  pathways  including  sensory 
cortex*  motor  cortex*  the  hippocampal  and  reticular  formations*  the 
cerebellum*  and  various  thalamic  nuclei.  It  is  not  known  whether  the 
diverse  set  of  subsystems  that  make  up  the  CHS  employ  general 
learning  mechanisms  that  are  few  in  number  or  several  specialised 
ones.  Two  plasticity  phenomena  motivate  the  ideas  in  the  present 
thesis: 

-  the  development  of  stimulus  selectivity  in  visual  cortex 
neurons 

-  potentiation  and  depotentiation  of  the  bilateral  pathway  from 
entorhinal  cortex  to  dentate  gyrus 

It  is  generally  agreed  that  the  instantaneous  spiking  frequency 
of  an  axon  reflects  a  spa tio temporal  integration  of  synaptic 
excitation  and  inhibition  over  the  corresponding  neuron*  but  the 
informational  content  of  a  neuron's  response  with  respect  to  the 
stimulating  activity  pattern  is  an  open  issue.  The  very  existence  of 
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behavioral  phenomena  implies  that  large  neural  networks  evolve  to 
give  appropriate  ensemble  responses  to  their  stimuli.  Bach 
constituent  neuron  must  therefore  develop  response  attributes  which 
are.  in  some  sense*  meaningful.  After  defining  the  model  neuron  and 
its  stimulus  environment  as  mathematical  structures*  Part  1  presents 
a  mathematical  framework  describing  two  complementary  forms  of 
abstraction  by  individual  neurons:  generalisation*  a  process  of 
finding  similarities  between  stimuli*  and  differentiation*  a 
separation  process.  An  analogous  dichotomy  in  cognitive  theory  has 
been  extensively  studied  by  Tversky  and  Gati  (1978*  1982).  They 
develop  a  “contrast  model"  of  similarity  and  difference  perception 
and  apply  it  to  psychological  data. 

Part  11  extends  a  theory  for  selectivity  development  in  visual 
cortex  proposed  by  Bienenstock  et  al  (1982)  which  describes  a  process 
by  which  cells  become  maximally  selective  over  their  input 
environments.  Development  of  orientation  specificity  and  ocular 
dominance  has  been  subject  to  extensive  study  using  many  paradigms 
that  manipulate  the  visual  environment  in  various  ways.  Some  of 
these  investigations  are  described  and  it  is  shown  that  the  theory 
accounts  for  the  corresponding  results.  It  is  then  shown  that  a 
simple  change  in  the  evolution  equation  describes  cells  which  seek 
minimum*  rather  than  maximum*  selectivity.  These  cells  tune  to 
cosmos  features  rather  than  distinctive  features.  Thus  a  general 
system  is  developed  to  account  for  both  types  of  selectivity 
development. 
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In  Part  III  an  alternative  model  is  introduced  to  describe  the 
same  set  of  visual  cortex  data.  It  also  includes  an  implementation 
of  the  maximum-minimum  selectivity  framework.  Here*  two  antagonistic 
mechanisms  are  postulated:  one  weakening  a  cell's  input  synapses  and 
the  other  strengthening  them.  These  mechanisms  are  coupled  by  a 
variable  which  evolves  according  to  the  same  equation  as  the 
synapses.  This  theory  rests  on  a  more  basic  set  of  hypotheses  which 
lead  to  a  mathematical  formulation  quite  similar  to  the  previous 
model's  starting  hypothesis*  hut  with  some  differences.  First*  the 
new  hypotheses  make  certain  assumptions  about  the  biochemical 
mechanisms  responsible  for  synaptic  modification  and  second*  there  is 
a  testable  difference  between  the  two  models  which  is  discussed  in 
Part  IV. 

Part  IV  describes  a  population  stimulus-record  method  for 
electrophysiological  testing  of  synaptic  modification  models.  The 
pre-  and  postsynaptic  activities  of  certain  synapses  can  be 
controlled  to  a  degree  and  their  efficacies  roughly  measured  in  some 
systems.  Such  measurements  can  give  important  information  regarding 
the  dependence  of  the  change  in  synaptic  strength  on  the  pre-  and 
postsynaptic  variables.  Since  this  method  is  applicable  to  neuron 
populations  it  may  prove  a  useful  tool  for  experimental  design  and 
data  interpretation.  The  predicted  results  of  the  model  in  Part  I 
are  discussed  as  well  as  those  of  some  other  models.  This  approach 
was  inspired  by  the  hippocampal  potentiation  studies  of  Levy  and 


Sc«rir4  (1979)t  so  it  is  no  accident  that  the  anatomy  and  physiology 
in  that  system  is  ideal  for  this  experimental  framework.  An  outline 

of  the  biology  of  the  entorhinal  cortex  to  dentate  gyrus  projection 
is  therefore  included  at  the  end  of  Part  IV. 


Background 


The  ability  of  organisms  to  process  information  constitutes  a 
common  basis  for  all  behavioral  phenomena.  Over  the  past  century* 
many  empirically  based  theories  of  behavior  have  evolved*  reshaping 
philosophical  and  clinical  approaches  to  mental  function.  A  nearly 
simultaneous  accumulation  of  discoveries  in  the  fields  of 
neuroanatomy  and  neurophysiology  has  established  a  detailed  picture 
of  the  nervous  system  as  an  enormously  complex  network  of  highly 
specialized  cells.  A  full  appreciation  of  neural  function  must  begin 
with  an  understanding  of  the  sub-microscopic  chemical  and  physical 
membrane  mechanisms  of  the  neuron  and  eventually  account  for 
psychological  data.  Theoreticians  are  attempting  to  close  the  gap 
between  neurobiology  and  psychology  by  developing  mathematical 
theories  of  learning  by  neurons  and  neural  networks. 

The  technique  of  stimulus-response  (SR)  is  perhaps  the  most 
valuable  protocol  in  all  of  experimental  science.  Response 
characteristics  may  provide  the  common  language  needed  to  relate 
electrophysiological  and  psychological  data.  We  will  assume  that 
the  action  taken  by  a  neural  network  is  dependent  solely  upon  the 
stimulus  applied  to  the  network  and  quantities  internal  to  the 
network.  These  internal  quantities  constitute  the  state  of  the 
network.  If  the  state  can  be  modified  by  the  application  of 
stimuli,  it  is  called  plastic.  Nonmodif iabie  states  are  termed 
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nonplastic.  Learning  processes  are  intrinsically  plastic  since 
they  involve  adjusting  response  properties  in  a  network.  This 
approach  to  the  study  of  learning  is  taken  by  Thorndike  (1932)  who 
begins  the  introduction  to  his  book  The  Fundamental  a  of  Learn ing 
with  this  sentence:  'We  are  concerned  in  this  volume  with  the 
fundamental  facts  of  learning  whereby  a  situation  which  first  evokes 
response  A  later  evokes  response  B.  different  from  A'. 

Note  that  nothing  has  been  said  regarding  the  complexity  of  the 
network.  A  neural  network  may  be  anything  from  an  entire  nervous 
system  down  to  a  single  neuron  or  even  part  of  a  single  neuron 
(Foggio  and  Koch.  1981).  If  the  network's  internal  quantities  and 
the  transfer  (input -out put)  function  are  known,  then  the  state  of  the 
network  has  been  specified  such  that  the  response  to  any  stimulus  can 
be  calculated.  Alternatively,  if  a  sufficient  (complete)  set  of 
responses  to  hypothetical  stimuli  and  the  transfer  function  are 
known,  then  the  internal  quantities  can  be  deduced,  again  specifying 
the  network  state.  Each  complete  set  of  neural  stimuli  is  a 
coordinate  system  for  the  network  state  space.  A  natural  or 
preferred  coordinate  system  is  given  by  the  internal  quantities.  A 
nonplastic  network  is  fully  described  by  its  state  and  transfer 
function.  However,  these  values  only  partially  specify  plastic 
systems. 

We  are  interested  in  the  dynamics  of  the  neural  state.  While 
the  state  and  transfer  function  represent  information  stored 
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order  of  10*)  sites  along  its  membrane.  These  sites  are  called 
synapses*  a  ten  coined  by  Sherrington  (1906)  to  indicate  a  junction 
betveen  neurons.  The  neuron  membrane  aaintains  a  "resting" 
electrical  potential  of  about  -60  mV  (interior  with  respect  to 
exterior).  Each  synapse  transduces  the  incident  signal  to  a  slight 
disturbance  in  the  resting  potential  across  the  membrane.  If  the 
soma  membrane  is  sufficiently  depolarized  (driven  above  about  -45 
mV),  a  pulse  is  transmitted  by  the  cell  along  its  axon.  The  pulse 
frequency  increases  vith  membrane  polarization  to  a  maximum*  while 
the  amplitude  remains  constant.  The  frequency-coded  signal  is  sent 
to  many  other  neurons  along  the  generally  intricate  arborization  of 
the  axon.  This  signal  then  either  inhibits  or  excites  other  cells  at 
synaptic  sites  by  hyperpolarizing  or  depolarizing  other  membranes. 

Thus  each  neuron  operates  as  a  stimulus-response  mechanism 
generating  a  single-valued  response  to  a  multidimensional  stimulus. 
Aggregations  of  nerve  cells  are  known  to  cooperate  functionally* 
receiving  inputs  from  and  projecting  to  common  populations.  So  the 
nervous  system  can  be  sequentially  broken  down  into  subsystems  of  an 
arbitrary  level  down  to  the  single-neuron  system.  The  state  of 
any  of  these  subsystems  can  be  characterized  either  by  its  SR 
properties  or  its  electrophysiological  properties  (internal 
quantities  and  transfer  function). 

Deterministic  treatments  of  neural  fraction  in  the  experimental 
framework  of  stimulus-response  methodology  are  well  suited 
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(  knowledge  )»  Che  storage  process  (“learning**)  is  expressed  by  a 
Modification  rule  or  plasticity  function  for  the  change  in  the 
neural  state*  This  rule*  together  with  the  stimulus  set  delivered 
to  the  network  thus  detemine  the  trajectory  of  the  system  through 
the  state  space. 

Let  us  now  consider  tbs  components  of  neural  networks.  Neurons 
are  the  basic  units  of  neural  function.  As  components  of  complex 
networks*  these  units  are  responsible  for  an  extraordinary  range  of 
information  processing  tasks  including  perception*  cognition*  and 
action.  Not  all  neurons  are  identical*  there  exist  several 
anatomically  defined  categories  of  these  cells  as  well  as 
corresponding  physiological  and  biochemical  properties,  but  they 
have  many  features  in  common.  All  neurons  contain  the  structures 
inherent  to  biological  cells*  however  they  are  not  self- 
replicating.  Furthermore*  each  neuron  transmits  a  time-varying 
electrical  signal  and  receives  several  simultaneous  electrochemical 
stimuli  which  also  vary  in  time. 

Nerve  cells  are  irregularly  shaped.  Extending  from  the  cell 
body  (soma)  are  several  processes  -  long*  narrow  extentions  of  the 
cell.  These  are  of  two  types:  dendrites*  which  generally  conduct 
signals  inwardly  toward  the  cell  body*  and  axons*  which  transmit 
frequency-coded  signals  to  other  cells.  A  typical  neuron  includes  a 
soma*  several  dendrites*  and  a  single  axon  which  may  have  several 
branches.  It  receives  signals  from  other  cells  at  many  (up  to  the 
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to  synaptically  based  theoretical  approaches*  The  81  technique  has 
been  used  by  both  physiologists  and  psychologists  for  many  decades. 
The  pioneering  works  of  Sherrington  (19061  and  Skinner  (1938) 
together  with  their  bibliographical  references  establish* 
respectively*  the  foundations  of  stimulus-response  methodology  in 
electrical  and  behavioral  contexts.  Both  experimental  approaches 
survive  today*  exemplified  by  electrophysiological  data  from 

i 

orientation-specific  cells  in  visual  cortex  (e.g.*  Hubei  &  Wiese 1* 
1962*  1977*  Rauschecker  &  Singer*  1981)  and  the  response  time  data  of 
subjects  learning  prototypes  (e.g.*  Posner  &  Reele*  1968*  1970). 

Several  mathematical  models  have  been  developed  to  account  for 
these  and  other  data.  Examples  of  such  theories  for  physiological 
data  are  Karr  (1969)*  von  der  Kalsburg  (1973)  and  Nass  &  Cooper 
(1973);  cognitively  based  models  include  Anderson  (1972*  1973)  and 
McClelland  &  Rumelhart  (1981).  Such  models  are  usually  based  on  the 
notion  of  "synaptic  weights**.  As  is  described  above,  a  neuron's 
instantaneous  activity  level  is  determined  by  a  spatiotemporal 
suanation  of  the  influences  of  the  activities  of  many  other  neurons. 
The  influence  of  one  neuron  upon  another  is  quantified  as  a  synaptic 
"weight**  or  "efficacy**.  These  synaptic  weights  along  with  other 
variables  generally  determine  the  transfer  function  of  a  neuron  or  a 
neuron  network. 

For  example*  a  linear  function  might  be  used  to  express  the 
output  y  of  a  neuron  to  its  inputs  x^  (1).  The  coefficients 


i 


•re  the  synaptic  weights*  representing  the  net  influence  of  one  cell 
upon  another.  This  can  easily  be  extended  to  a  network  consisting  of 
a  population  (Xj)  projecting  to  a  population  {y^>  (2).  A  simple 
nonlinear  example  is  the  threshold  function  given  by  (3>*  each 
neuron  y^  having  an  activity  threshold  8^ . 

y  -  £ai*i  (1) 

i  x  x 

»i  ■  «> 

-  V  <3> 

tjhere  K(x)  s  {j 

Such  matrix  or  associative  approaches  began  to  receive  attention 
in  the  1940's  (McCulloch  and  Pitts*  1943s  Culbertson*  1948).  This 
sort  of  siathematical  formulation  makes  clear  the  close  relationship 
between  the  transfer  function  and  the  quantification  of  the  neural 
state.  Modification  rules  (differential  equations  for  the  change  in 
u  or  e)  are  formulated  or  hypothesised  in  models  for  learning.  An 

early  hypothesis  which  has  been  a  guiding  principle  for  many 
theories  is  Hebb's  (1949)  “neurophysiological  postulate”: 

“When  an  axon  of  cell  A  is  near  enough  to  excite  a 
cell  B  and  repeatedly  or  persistently  takes  part  in  firing 
it.  some  growth  process  or  metabolic  change  takes  place  in 
one  or  both  cells  such  that  A's  efficiency*  as  one  of  the 
cells  firing  B*  is  increased. “  Cp.  82] 


a 


This  states  for  neural  function  what  Thorndike* a  (1913)  Law  of 
Use  states  for  learning  behavior: 

"The  Law  of  Use  is:  When  a  Modifiable  connection  is 
made  between  a  situation  and  a  response*  that  connection's 
strength  is*  other  things  being  equal*  increased.”  [p.  2] 

"Ultimately  degrees  of  strength  of  a  connection  in 
behavior  will  be  defined  as  degrees  of  some  anatomical  or 
physiological  fact  whereby  synapses  between  neurones  differ 
in  intimacy."  [p.  3] 

Most  synaptically  based  learning  schemes  extend  this  assumption 
to  account  for  decrease  in  syntaptic  efficacy  as  well.  Inhibitory 
synapses  must  be  separately  considered  as  in  Stent  (1973)  and  Levy 
and  Desmond  (1982).  An  extensive  examination  of  learning  rules  and 
their  application  is  provided  by  Xohonen  (1977). 

The  degree  to  which  memory  is  distributed  is  an  issue  addressed 
by  many  models.  Lashley  (1929)  sought  to  locate  the  site  of 
learning  in  rat  cortex  by  performing  an  extensive  series  of  cortical 
lesions  and  measuring  the  ability  to  learn  maze  patterns  as  a 
function  of  the  extent  and  location  of  each  lesion.  His  results 
indicate  that  this  ability  is  not  localized  to  any  particular  locus 
in  cortex: 


"The  same  retardation  in  learning  is  produced  by  equal 
amounts  of  destruction  in  any  of  the  cyto-architectural 
fields.  Hence  the  capacity  to  learn  the  maze  is  dependent 
upon  the  amount  of  functional  cortical  tissue  and  not  upon 
its  anatomical  specialisation."  Cp.  175] 
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Lashley's  data  are  summarised  in  Figure  1. 

The  notion  of  distributed  memory  is  attractive  to  many 
theoreticians.  A  bolograa  model  was  put  forward  by  Longuet-liggina 
(1968)  since  holography  is  a  non-local  storage  technique  —  partial 
destruction  of  a  bolograa  reduces  overall  resolution  of  the  stored 
image  without  affecting  any  particular  region. 

The  opposing  viewpoint  is  held  by  many  researchers  who  point  to 
observations  (e.g.  Hubei  and  Hiesel*  1963)  of  feature  sensitive 
neurons  in  cortex.  Several  classes  of  selective  neurons  have  been 
defined  according  to  their  modality  and  degree  of  specificity  (Figure 
2).  The  level  of  neuronal  specificity  at  the  srnst  “gnostic”  stage  is 
thought  to  indicate  the  extent  to  which  memory  and  cognition  are 
localized.  Barlow  (1972)  postulates  the  “cardinal  cell”  as  a 
compromise  between  Sherrington's  (1940)  “pontifical  cell~  concept  and 
complete  distribution  of  memory  traces. 

Recent  experimental  data  demonstrate  even  higher  specificity 
than  the  well-documented  hypercomplex  neurons  in  visual  cortex. 

Bruce  et  al  (1981)  find  polymodal  neurons  in  the  extrastriate  area 
they  call  the  superior  temporal  polysensory  area  (STP).  Gross  et 
al  (1972)  report  neurons  in  macaque  inferotemporal  cortex  selective 
to  visual  inputs  resembling  a  monkey's  hand. 


Figure  2 .  Thp  hierarchical  Model  of  the  visual  iTitw.  Neuron 

classes  are  labelled  on  the  right  with  corresponding  receptive 
fields  on  the  left.  (Fro*  von  Fieandt  &  Moustgaard  (1977)) 


Figure  1 .  Lgghl»g»«  Ltihley'i  data  ihovi  «  roughly 

continuous  dependence  of  problsa  solving  espsbility  on  the 
difficulty  of  the  problea  end  on  the  degree  of  injury  to  the 
brain.  C From  Lash ley  (1929)) 


Finally,  experiments  by  Yeo  at  si  (1982)  and  Clark  et  al 
(1982)  identify  specific  cerebellar  nuclei  that  abolish  classically 
conditioned  responses  in  rabbits.  These  may  be  the  loci  sought  by 
Lashley.  It' seems  clear  that  the  brain  is  specialized  into 
functional  regions,  however  the  internal  operation  of  these  areas 
may  be  distributed  to  some  extent.  Also,  the  neural  loci  of 
individual  memory  traces  stay  be  distributed  over  several 
functionally  distinct  areas. 


JU  zbfi  Theoretical  Fraanrnrk  8  Definitions  A&d  CcBcmti 


L.  The.  Mod*i  Heurca  aad  feral lei  Modification 


It  is  assuaed  that  each  neuron  generates  a  dynamic  frequency- 
coded  scalar  response  to  a  time-varying  vector  stimulus  and  that  the 
response  properties  evolve  with  experience  (learning).  Each  of  N 
afferent  signals  d^  are  transduced  to  membrane  depolarizations  and 
ultimately  induce  a  potential  (x)  at  the  soma.  Spatiotemporal 
integration  is  idealized  here  to  be  an  instantaneous  linear  sum  x  of 
the  N  inputs  d^  weighted  by  the  corresponding  N  synaptic  efficacies 
mj  (4). 


N 

*  -  E®!*! 

i  X  X 


(4) 


For  now*  the  integrated  activity  level  x  may  be  considered  to 
correspond  to  an  arbitrary  physiological  measure  of  the  cell’s 
excitation  level  such  as  the  net  postsynaptic  potential  or  the 
resulting  firing  frequency.  Although  it  is  somewhat  important  for 
the  signals  to  be  summed  linearly,  the  neuron’s  output  frequency  may 
be  described  by  certain  (nondecrcasing.  positive,  bounded,  and 
continuous)  nonlinear  i unctions  of  x.  Thus  the  response  of  the  model 


neuron  is  purely  a  function  of  its  synaptic  weights 


The  state  of  the  neuron  is  specified  by  the  synaptic  vector 
«£  together  with  an  internal  quantity  q  which*  although  it  plays  no 
role  in  the  transfer  function  (4)»  modulates  the  modification  or 
plasticity  process  (5).  Such  systems  are  (N+l) -dimensional:  N 
synapses  plus  the  internal  quantity. 


■^(t)  -  $(x(t) ,q(t))di(t)  (5a) 


q(t)  -  i|;(x(t)  ,q(t)) 


(5b) 


Since  (5b)  is  an  autonomous  equation*  q  reflects  a  temporal 
integration  of  x  over  some  recent  interval*  the  scope  of  which  is 
determined  by  the  relative  magnitudes  of  ♦  and  Thus  the  synaptic 

vector  m  is  modified  parallel  or  antiparallel  to  the  presented 
stimulus  vector  d*  the  strength  of  the  modification  being  a  function 
of  x  and  q.  Of  all  possible  input  stimuli  of  a  given  magnitude* 
the  response  to  the  one  in  the  direction  of  d  is  most  greatly 
affected  by  the  change  m.  Parallel  modification  thus  has  the 
following  behavioral  implication:  learning  induced  by  g  particular 
stimulus  more  greatly  influences  future  reBPPPaea  that  ALilllU 
than  to  others  of  the  a  amp  magnitude.  The  intuitive  plausibility  of 
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this  statement*  though  it  may  not  rigorously  hold*  supports  the 
formulation  (5).  One  must  be  aware  that  "magnitude**  is  a  nebulous 
term  in  this  context. 

Certain  functions  $  and  4i  that  yield  otherwise  desirable 
attributes  for  describing  certain  types  of  neural  plasticity*  may 
cause  some  or  all  of  the  state  components  (m^,q)  to  grow  without 
bound.  There  are  several  standard  approaches  to  problems  of 
divergence.  For  example  the  sum  of  the  synaptic  weights  may 

be  normalized  to  or  restricted  to  be  less  than  a  particular  value. 
Another  approach  is  to  introduce  an  exponential  decay  term  (6J.  While 
it  i6  not  included  in  the  bulk  of  this  thesis.  Appendix  A  illustrates 
the  utility  of  this  term  to  a  specific  application. 

m(t)  »  $d  -  e(m)  (6) 


Om.  Hut  SLinuIm.  Eaviioiiaeat 

Like  the  term  "’neuron”*  the  "stimulus  environment”  is  a  tangible 
to  the  experimentalist  and  an  abstract  mathematical  entity  to  the 
theoretician.  All  possible  neurally  encoded  stimuli  d  can  be 
represented  by  points  in  a  hypervolume  U  (universe)  in  N-space 
bounded  along  each  coordinate  by  a  minimum  firing  frequency  (zero) 
and  a  maximum*  determined  by  the  absolute  refractory  period  of  the 


i 
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afferent.  A  world  which  presented  input  stimuli  from  a  uniformly 
random  distribution  over  the  entire  hypervolume  would  be  completely 
chaotic  -  a  maximum  entropy  environment  containing  no  information 
(snow  on  television  and  radio  static  are  electronic  examples  of  high 
entropy  environments).  The  complexity  of  an  environment's  information 
content  is  reflected  by  the  topological  structure  of  the  input 
environment.  An  environment  may  be  any  probability  distribution  over 
the  entire  stimulus  space*  continuous  or  discrete. 

Sensory  experience  of  all  modalities  presents  the  healthy 
organism  with  a  highly  structured  set  of  stimuli.  In  the  course  of 
processing  information*  the  network  codes*  recodes*  integrates*  and 
finally  generates  a  response  to  these  stimuli  and  it  must  learn  to 
respond  appropriately.  The  precise  nature  of  the  coding  will  not  be 
addressed  here.  The  stimulus  sets  used  in  computer  simulation  were* 
in  general*  "fuzzy"  (having  a  small  random  component)  subspaces  of  U 
that  share  certain  topological  properties  with  the  physical  cumuli 
used  in  the  laboratory. 

The  order  of  presentation  of  patterns  from  the  environment  is 
given  by  a  stochastic  process*  which  may  be  either  discrete  or 
continuous.  In  the  present  thesis*  a  stationary  jump  process  of 
independent  presentations  is  assumed  for  most  of  the  analysis  and 
computer  simulation.  The  trajectory  profile  imposed  by  the  process 
is  illustrated  by  averaging  the  system  (5)  over  the  stimulus 
environment  E  to  generate  a  deterministic  process  (7)  yielding 
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solutions  for  (m£(t).q(t))  from  various  initial  states 
<“i(t0>.q(t0)). 

<mi(t)>E  -  <«(x(t),q(tX>d1(t)>E 

-  /£  ♦(x(t),q(t))di(t)  P(dE)  (7) 

<q(t)>E  -  <t|»(x(t)  ,q(t))>t  -  JE  4»(x(t)  ,q(t))  P(dE) 


Corrupt  EnY,iro,Dafiat.a. 

Generally  a  stimulus  space  is  neither  entirely  noisy  nor  is  it 
devoid  of  noise  --  it  contains  information  corrupted  by  random  or 
uncorrelated  interference.  The  separation  of  a  signal  from  static 
can  be  a  tedious  task,  but  in  general  a  repeated  signal  reinforces 
itself  while  noise  interferes  with  itself  destructively.  Thus, 
systems  such  as  those  discussed  in  this  thesis  are  resistant  to  noise 
given  sufficient  exposure  to  the  stimulus  environment.  Interestingly, 
noise  can  enhance  the  development  of  neural  networks  (Anderson  et 
al.  1977).  This  is  discussed  in  Part  Ila. 

In  the  following  sections,  stimuli  will  be  generated  as  the  sum 
of  patterned  and  chaotic  components.  The  pattern  space  E  can 
therefore  be  considered  a  k-dimensional  manifold  (k<N)  in  the  N- 
diroensional  space  U.  The  environment  or  stimulus  space  D  is  the 
sum  space  of  the  pattern  space  and  a  more  homogeneous  process.  One 
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can  imagine  the  stimulus  space  as  an  N-dimensional  "fuzz"  encasing 
the  pattern  subspace  (Figure  3). 


Selectivity 

A  neural  response  function*  c(m.E),  can  be  calculated  over  the 
pattern  set  E  for  a  given  synaptic  state  m.  Bienenstock  (1980)  gives 
a  precise  definition  of  selectivity  for  a  general  stimulus  density  in 
terms  of  such  a  density  function.  For  the  purpose  of  this 
presentation*  this  is  paraphrased  (8). 


Sel(m.E) 


/P  c(m,E)  P(dE) 

1  -  -H - 

cmax 

0 


where  c  S  max(c(m,E)) 
max  _ 


c 

max 


>  0 


(8) 


c  ■  0 
max 


Note  that  Sel  (m,E) €  [0*1]  with  0  indicating  a  uniform  response 
across  E  and  1  indicating  that  the  entire  set  of  evocative  stimuli  is 
of  measure  aero  with  respect  to  E. 


Ei£UX£  i*.  4  ixa.  dimensional  ~fuzzv"  stimulus  apace.  The  union  of 
the  shaded  regions  is  a  stimulus  space  consisting  of  a  pattern 
space  {d^.d^}  and  a  random  vector  uniformly  distributed  on  a 
small  circular  srea. 
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Circular  and  other  periodic  environ— nta 

A  pattern  set  E  is  called  circular  if  it  describes  a  circle  of 
uniform  density  embedded  in  a  sphere  about  the  origin.  That  is  the 
elements  d(v)  of  E  must  satisfy  |d(w)^  =  constant*  p(v)  =  ^  and 
d(vj)  .  d(w2)  =  f(w2-wj)  where  w  is  a  parameter  of  the  period 
T  and  p(v)  is  a  density  function,  A  circular  lattice  pattern  set 
ia  an  analogous  structure  with  finitely  many  elements  equally  spaced 
on  a  circle.  The  response  characteristics  of  a  synaptic  state  with 
respect  to  such  one-parameter  entities  can  be  represented  by  a  simple 
function  of  one  variable  or  index  from  which  the  selectivity  is 
easily  evaluated  (Figure  4). 

Circular  environments  are  idealisations  of  test-pattern  sets 
used  by  physiologists  to  investigate  orientation  specificity 
development  in  visual  cortex.  The  neural  representations  of 
oriented  contrast  bars  of  constant  width  certainly  constitute  a 
closed  one-parameter  (i.e.*  orientation)  curve*  if  not  a  circle*  in 
the  axon  frequency  space  of  the  optic  nerve.  Indeed*  data  from  such 
experiments  is  plotted  on  a  so-called  orientation  tuning  curve  much 
like  the  response  curves  in  Figure  4.  Thus*  response  data  and 
selectivity  measurements  over  them  represent  a  point  at  which  theory 
and  experiment  meet. 

General  and  arbitrary  circular  environments  of  high  dimension 
for  computer  simulation  are  not  straightforwardly  generated.  This 


ction  of  the  rectangle  that  is 
ivity  of  the  response  curve. 


Motivates  retinal  array  pattern  mapping  (illustrated  in  Figure  5)* 

•  straightforward  technique  for  simulating  retinal  coding  of  any 
spat iotemporal  visual  stimuli.  An  array  is  assembled  of  M  units  of 
uniform  site  and  shape  constituting  the  model  retina*  on  which 
geometric  stimuli  are  projected.  The  i-th  component  of  the  pattern 
vector  is  a  measure  of  the  "excitation  density"  of  the  i-th  retinal 
unit.  Retinal  mapping  distorts  geometrical  relationships  between  the 
visual  stimuli*  but  fortunately  the  topology  is  conserved.  Thus* 
applying  uniform  brightness  patterns  of  constant  width  and  intensity 
to  such  an  array  results  in  a  pseudocircular  pattern  set  for  finite 
N. 

Circular  and  pseudocircular  environments  are  adequate  for 
simulating  one-  parameter  patterns  coded  by  one  retina.  They  suffice 
as  well  for  cells  receiving  perfectly  correlated  patterns  from 
several  retinae.  However  more  general  treatments  of  multiocular 
coding  of  circular  visual  environments  require  that  the  destination 
(neural)  map  be  a  K-parameter  product  apace.  Binocular  experimental 
paradigms  invoking  different  stimulus  sequences  to  the  two  eyes* 
such  as  strabismus  or  binocular  disparity*  can  be  simulated  using 
toroidal  or  pseudotoroidal  environments*  sets  of  patterns 
characterised  by  two  independent  periodic  parameters. 


Figure  5 .  Retinal  array  pattern  generation.  The  inter aect iona  of 
oriented  bars  with  the  cells  in  a  simple  geometric  array  was 
used  to  generate  pattern  vectors  for  computer  simulation.  The 
technique  is  inspired  by  the  principle  of  the  retina*  but  it  is 
not  intended  to  be  a  precise  retinal  model  since  the  inputs  are 
used  for  a  model  neuron  in  cortex. 
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&L  HcuiqmI  Abstraction 

Each  of  the  models  presented  in  this  thesis  describes  neurons  of 
two  kinds*  which  differ  in  the  nature  of  the  information  they 
abstract  from  their  stimulus  environments.  Neurons  which  seek  to 
minimize  their  selectivity  functions  will  be  called  similarity 
calls  or  S-cells.  since  they  seek  the  pattern  component  most 
common  to  their  environmental  stimuli.  The  other  kind  of  neuron* 
seeking  maximum  selectivity*  will  be  labelled  a  diffaranca  cell  or 
D-cell.  These  two  functions  should  be  considered  elementary 
abstraction  processes  into  which  more  complex  functions  can  be 
analyzed. 

Selectivity  Magimizatioa 

Consider  an  environment  of  two  stimuli  represented  by  vectors 
d*  and  A2,  The  selectivity  S(m,d*,d^)  for  the  linear 
transfer  function  (4)  is  given  by  (9). 


S(m*d1,d2) 


.1  A.  A2 

m*d  +  m*d 

2tnax(tn-  d*) 

1 


(9) 


Lines  of  constant  selectivity  can  be  plotted  (Figure  6)  in  the 
plane  spanned  by  two  patterns.  Maximally  selective  (Ssl)  states  are 
orthogonal  to  one  vector  and  not  to  the  other.  Minimally  selective 
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(S=0)  state*  are  orthogonal  to  the  difference  vector  d^  -  d*. 

For  arbitrary  S,  m  must  satisfy  (10). 

I(2S  -  Dd1  +  dJ].»  -  0 
where  m*d*>  a*d^ 

The  goal  of  each  model  is  to  drive  the  states  of  S-cells  to  an 
S=0  point  and  D-cells  to  an  S=1  point.  Constraints  on  the  synapses 
(such  as  the  sign  of  m)  may  prevent  a  cell  from  realizing  optimal 
selectivity.  In  such  cases*  the  neuronal  state  will  be  driven  to 
the  constraint  boundary. 

Evidence  tax.  Hut  development  at  neuronal  abatxaction  proper tie»  in 
viaual  corien 

Single  unit  recording  techniques  were  employed  by  Hubei  and 
Uiesel  (1959)  to  reveal  the  highly  specific  response  properties  of 
neurons  in  cat  striate  cortex.  They  found  that  these  cells  are 
sensitive  to  the  orientation  of  contrast  edges  and  bars  in  their 
visual  fields.  These  neurons  were  shown  (Wiese 1  and  Hubei*  1963)  to 
develop  their  selectivity  attributes  over  the  first  ten  weeks  after 
birth  ~  the  so-called  critical  period.  The  development  can  be 
influenced  by  manipulating  the  kitten's  visual  environment  during  the 
critical  period.  Results  of  various  laboratory  paradigms  for  such 
manipulations  are  now  described. 


normal  Rearing  (NR):  Kittens  reared  in  normal  visual 
environments  develop  (for  the  most  part)  neurons  which  are  highly 
sensitive  to  orientation  (Hubei  and  Wiesel*  1959*  1962).  Layer  IV 
c*ll*^  develop  to  be  responsive  to  inputs  from  one  eye.  Other 
cells  exhibit  binocular  responses  which  may  favor  one  eye  or  the 
other.  The  preferences  for  ocularity  and  orientation  vary 
topographically  across  cortex. 

Monocular  Deprivation  (MD):  Neurons  in  the  visual  cortex  of  a 
kitten*  deprived  of  visual  stimulus  to  one  eye  during  the  entire 
critical  period  by  means  of  eyelid  suturing*  become  generally 
nonresponsive  to  inputs  presented  to  the  deprived  eye  (Blakemore. 
1976). 

Beveran  Suture  (RS):  The  effects  of  the  MD  paradigm  can  be 
reversed  during  the  critical  period  by  opening  the  closed  eye  and 
closing  the  open  one  (Movshon.  1976). 

Dark  Rearing  (DR):  One  would  probably  intuit*  based  on  the  NR 
and  MD  results*  that  binocular  deprivation  would  cause  complete  non- 
responsiveness  in  visual  cortex  neurons.  Moreover*  while  the  ocular 
dominance  statistics  indicate  a  shift  toward  monocularity  (Blakemore 
and  Van  Sluyters*  1975)  many  cells  are  binocularly  responsive 

*  Nearly  all  layer  IV  neurons  receive  direct  thalamic  input 
(Hornung  and  Garey*  1981). 


(Leventhal  and  Hirsch,  1980)  and  very  few  cells  ere  non-responsive 
(Buisseret  and  Imbert.  1976). 

Artificial  Strabismus  (AS):  By  cutting  one  of  the  extraocular 
muscles  in  newborn  kittens.  Hubei  and  Wiesei  (1965)  eliminated  the 
normally  high  correlation  between  inputs  from  the  two  eyes.  They 
found  most  cells  developed  monocular ly. 

Each  of  these  experiments  support  the  conjecture  that  some 
neurons  seek  maximum  selectivity  over  their  stimulus  environments. 
The  search  for  cells  of  low  selectivity  is  less  intense  and  hence 
the  data  less  abundant.  Nonetheless,  cells  in  visual  cortex  have 
been  classified  as  “circularly  symmetric**  (Hubei  and  Wiesei.  1977). 
Other  authors  (e.g.»  Kelly  and  van  Essen,  1974)  report  difficulty  in 
finding  a  cell's  optimal  stimulus.  Barlow  (1972)  pointB  out  that 
many  neurons  display  invariance  in  their  response  as  a  stimulus 


changes 
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ULjl  Generalization  aaA  Differentiation  by.  Single  MeurOHB « 

IS itt  Approaches 


Ljl  Ibe.  Fixed  Point  Model 

In  this  section  a  version  of  the  model  put  forward  by 
Bienenstock  et  al  (1982)  for  development  of  orientation  specificity 
in  visual  cortex  is  described  and  then  extended  to  include 
selectivity-minimizing  neurons  as  well.  Bienenstock' s  (1980)  doctoral 
thesis  includes  formal  theorems  and  detailed  proofs  in  support  of 
that  theory.  The  extended  model  presented  in  this  section  will  be 
called  the  fixed  point  (E£)  model  to  distinguish  it  from  the 
alternative  formulations  discussed  in  subsequent  sections.  The  FP 
model  (and  those  that  follow  as  well)  uses  parallel  modification 
and  hence  can  be  expressed  in  terms  of  (A)  and  (5). 

Selectivity  maximization 

The  function  <t>  for  the  maximum-selectivity  (D-cell)  process  must 
be  continuous.  bounded,  and  satisfy  (11) 

■  *gn(x)  sgn(x  -  qP) 

(11) 

where  p  >  1 


The  rate  of  change  of  q(t)  is  given  by  (12).  This  is  chosen  so  that 
q  is  a  running  time  average  over  the  activity  x(t)  (13). 


4»D(x,q) 


B(x  -  q) 


B>  0 


(12) 


q(t) 


exp[-S(t-tQ) ]q(tQ)  +  B 


|  exp[-B(t '  -  tQ)]x(t')  dt' 
*0 


lllfi.  (1+D  -  Dimensional  System 

Consider  a  simple  system  of  a  single  synapse  m  receiving  a 
steady,  noiseless  signal  d=l .  The  concept  of  selectivity  is 
meaningless  with  respect  to  a  one-element  pattern  environment,  but 
aspects  of  the  system  dynamics  are  easily  analyzed  in  this  two- 
dimensional  state  space.  The  system  is  a  completely  deterministic 
pair  of  coupled  autonomous  equations  for  m  and  q  (14). 

m(t)  -  $D(m(t) ,q(t) ) 

q(t)  -  B(m(t)  -  q(t) )  (14) 

m(0)  >  0  ;  q (0)  >  0 

The  critical  points  of  the  system  lie  at  the  intersection  of  the 
nullclines  m=0  and  q=0  (Figure  7),  which  partition  the  state  space 
into  regions  characterized  by  the  signs  of  the  state  velocity 
(sgn  m,  sgn  <}),  the  quadrant  of  (m,q). 

The  asymptotic  behavior  of  (14)  is  never  divergent  (Theorem  1) 
Depenaing  on  the  value  of  B»  the  trajectory  (m(t),q(t))  either 
converges  to  (1,1)  or  to  a  finite  limit  cycle  (Figures  8a  and  8b). 


(13) 


jgure  8.  Computer  generated  jjra  jectorifi&«  The  nu 11c lines  from 

Figure  7  are  again  shown  along  with  sample  trajectories  for  two 
values  of  0.  (a)  0  =  0.7  gives  limit  cycle  behavior  --  two 

trajectories  are  shown:  one  spirals  in  from  the  outside  and  the 
other  from  the  inside.  <b)  B  =  1.3  gives  convergence  to  the 
point  (1»1). 
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Iheorea  1  AS  t-*»,  the  system  (14)  drives  the  state  to  either  the 
origin,  the  point  (1.1).  or  to  a  closed  path  about  that  point* 

Proof  Since  (14)  is  an  autonomous  system*  (m*$)  is  purely  a 
function  of  (m.q)  and  therefore  no  trajectories  cross  at  any  point 
vith  the  possible  exception  of  a  critical  point  ((m.q)  =  (0.0)). 
Trajectories  may  only  intersect  at  a  critical  point  in  the  limit  as 
t-*« . 


Along  the  coordinate  axes.  (14)  simplifies  to  (15)  and  (16) 


»(ra*0)  ■  0 
^(m-0)  •  -Bq 


m(q-0)  -  4(m(t),0)>  0 
q(q-0)  -  8m 


(16) 


Therefore  if  m(0)>0  and  q(0)>0  then  m(t)>0  and  q(t)>0  for  all 
t>0  (see  figure  8).  Nov  consider  the  four  regions  defined  by  the 
nu 11c lines: 


Rj  :  (sgn  m*  sgn  q)  =  (+lt+l) 
R2  s  (sgn  m.  sgn  q)  =  (-1.+1) 
R3  s  (sgn  m*  sgn  q)  =  (-1,-1) 
R4  :  (sgn  m,  sgn  q)  =  (+1,-1) 
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In  Rj  q>0»  but  a  trajectory  may  not  cross  from  Rj  to  R4. 

Since  ♦  is  bounded*  a  state  in  R^  either  asymptotically  approaches 
(1*1)  or  is  driven  to  R2.  A  trajectory  in  R2  must  cross  into 
R3.  In  R3,  each  state  is  driven  toward  the  q-axis  and  toward 
R4.  However  since  the  q-axis  is  a  trajectory  to  (0*0)*  a  state  in 
Rj  must  either  enter  R^  or  go  asymptotically  to  (0*0). 

A  linear  approximation  (17)  shows  that  if  $B(0,0)  >  0*  the 
origin  is  a  saddle  point*  and  hence*  unstable.  In  this  case*  only 
the  trajectory  exactly  along  the  q  axis  is  not  pushed  into  Rj . 


m(sr*0,  q-+0)  »  d  m 
m 

$  -  B(m-q) 


(17) 


The  stability  of  the  critical  point  at  the  origin  depends  on  the 
derivative  of  4  and  the  magnitude  of  B.  We  will  use  a  sufficiently 
well  behaved  and  a  sufficiently  high  value  of  6  to  assure  convergence 
to  (1.1). 

Xh£  JL2+11  dimensional  D,-.Cell 

Now  consider  a  cell  with  2  synapses  in  an  environment  consisting 
of  exactly  two  independent  patterns  (d^.d^).  This  is  the 


simplest  environment  in  which  selectivity  can  be  measured.  The 
corresponding  stochastic  system  o£  coupled  ODE's  (18)  possesses  4 
equilibrium  points  at  which  (si^.q)  =  0  for  both  stimulus  vectors. 

rn^t)  -  dpCm^d.q)^ 

AjCt)  -  4D(m*d,q)c[2  (18) 

q(t)  ■  6((m.d)-q) 

1  2 

for  the  random  stimulus  <J«(d  ,d  } 

Since  the  input  stimuli  are  independent*  this  condition  is 
prerequisite  to  the  more  general  deterministic  criterion  <m£*q>Q=0. 
Two  of  the  equilibrium  points  have  zero  (minimum)  selectivity.  The 
other  two  each  have  selectivities  of  0.5*  the  maximum  for  this 
environment.  One  of  these  is  preferential  to  d*,  the  other  to 
d2.  A  projection  of  the  system  to  the  mi-u^  plane  (Figure  9) 
illustrates  the  situation.  The  point  m  =  (1*1)  is  a  saddle  point  and 
m=(0.0)  is  an  unstable  node.  Only  the  points  of  maximum  selectivity 
are  stable.  A  detailed  analysis  of  a  very  similar  system  is  done  by 
Bienenstock  (1980). 


Kishex  Pineagioaa lilies 


The  (N+l)  dimensional  generalization  (19)  admits  2N 
equilibrium  points  in  an  environment  of  N  independent  stimuli 


Figuns  Sjl.  Projection  al  the.  L2±ll  Pre-fill  into  the  a-Rlaag»  The 

figure  shows  an  idealized  two-dimensional  system  in  which  q  is 
the  expected  value  of  m*d  evaluated  over  the  entire  environment 
The  parabolas  are  nullclines  corresponding  to  each  of  the 
patterns:  m*di  =  q2  and  m»d2  =  q2# 


*j(t)  -  $D(m.<i,q)d, 

$(t)  “  8((m*d)-q) 


(19) 


for  Che  random  stimulus  jdC(d^> 

Of  these*  N  are  maximally  selective*  with  selectivity  (N-l)/M*  tvo 
(the  origin  and  (uij . .n^.q)  =  (1.....1.1))  exhibit  zero 
selectivity*  and  the  remaining  2N-H-2  display  intermediate  levels 
of  selectivity.  Each  of  the  maximally  selective  points  is  "tuned**  to 
a  different  one  of  the  N  patterns.  It  is  conjectured  on  the  strength 
of  numerical  simulations  (Figure  10a)  that  only  these  points  display 
asymptotic  stability.  This  has  been  proven  for  a  similar  system  in 
an  orthogonal  environment  (Cooper  et  al*  1982). 


PfiBfindent  Stimuli 

Environments  containing  more  than  N  stimuli  also  drive  the  model 
neuron  (19)  to  high  selectivity  (Figure  10b).  The  maximum 
selectivity  attainable  is  a  function  of  the  environment.  In 
particular  this  is  sensitive  to  the  extent  of  separation  between 
vectors  in  the  pattern  set  (Bienenstock*  1980). 

One  can  generally  consider  9  as  a  function  that  "reinforces** 
patterns  giving  a  high  response  (x>q)  and  "suppresses"  patterns  that 
evoke  low  (x<q)  activity  (Figure  11). 


Figure  IQ.  Tuning  curves.  Computer  simulation  shows  that  the 

system  of  differential  equations  given  by  (19)  drive  the  model 
neuron  to  maximum  selectivity,  (a)  Independent  stimuli:  all  but 
one  of  the  patterns  give  a  null  response  in  the  final  state. 

(b)  Dependent  stimuli:  some  patterns  give  intermediate 
responses. 


INTEGRATED  POSTS 


SELECTIVITY  MAXIMIZATION 


8(q,)/'eiq2) 


<D  ®  0  0  CD 

ORIENTATION 


EiftUifi.  LLu  IkeL  IhflSll  in  K  dimensions.  The  function  *D(x,q)  is 

shown  (right)  plotted  against  x  for  two  values  of  q.  The  effect 
of  ^..-modulated  parallel  modification  is  illustrated  on  the 
left. 


fcg;  witte.:.--:- 
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comparison  q±  tile,  model  with  various  binocular  interaction  paradigms 

The  visual  system  is  perhaps  the  best  understood  of  mammalian 
sensory  systems.  This  model  is  in  good  agreement  with  the  bulk  of 
data  for  binocular  selectivity  development  in  visual  cortex.  Belov* 
the  experimental  paradigms  described  in  Part  Ic  are  discussed  in 
terms  of  the  model.  Both  analysis  and  computer  simulation  support 
the  agreement  between  theory  and  experiment. 

Normal  Rearing  (NR):  The  NR  stimuli  are  represented  as  2N- 
dimensional  vectors  in  which  the  N  left-eye  components  are  highly 
correlated  with  the  N  components  from  the  right  eye.  The 
environment  is  essentially  circular  since  the  patterns  are  periodic 
over  a  single  parameter.  Results  from  computer  simulation  are  shown 
in  Figure  12a. 

Binocular  Deprivation  (BD):  Patternless  inputs  ("pure"  noise) 
drive  the  state  of  a  model  neuron  through  a  random  walk.  Thus*  the 
state  wanders  aimlessly  from  its  initial  condition.  Figure  12b 
illustrates  a  typical  result  from  computer  simulation. 

Monocular  Deprivation  (MD)  and  Bgyeggg  Suture  (RS): 


Deprivation  of  patterned  input  to  a  single  eye  by  way  of  eyelid 
suture  results  in  a  large  shift  in  ocular  dominance  favoring  the 
functional  eye.  The  theoretical  explanation  for  the  MD  and  RS 


Ei&Uie  12a.  CflBPUtfil  simulation  ol  four  laboratory  paradigms 
text  for  description. 


phenomena  is  subtler  than  for  MR  and  BD  and  is  discussed  below.  The 
MD  environment  is  represented  by  a  2N-dimensional  vector  generated  by 
concatenating  N  noise  components  to  an  N-dimensional  pattern  (or 
pattern-plus-noise)  vector.  Note  that  computer  simulation  (Figures 
12c  and  12d)  supports  the  success  of  the  model  in  accounting  for  the 
experimental  results. 

Artificial  Strabismus  (AS):  The  AS  environment  is  toroidal  (or 
pseudo-toroidal),  being  periodic  over  two  independent  parameters, 
namely  the  left-eye  orientation  and  the  right-eye  orientation.  Thus 
the  AS  inputs  are  2N-dimensional  concatenations  of  independently 
selected  patterns.  Computer  simulation  suggests  that  only  monocular 
states  are  stable.  Figure  12c  (MD)  reflects  a  typical  AS 
simulation.  Occasional  trials  show  binocular  selective  states  (the 
preferred  orientations  of  the  two  eyes  appear  to  be  uncorrelated) 
that  survive  for  some  time  but  eventually  become  monocular.  Figure 
13  shows  the  final  state  plotted  over  a  toroidal  environment. 

Oxigntaiiao  S&l£.cj.iYity  aaA  Qsulai  Dominance 

Detailed  examination  of  theoretical  behavior  in  an  MD 
environment  highlights  a  subtle  relationship  between  orientation 
selectivity  and  ocular  dominance.  Here  it  is  explained  why  on  the 
one  hand  deprivation  drives  synapses  to  zero  efficacy  in  an  MD 
environment*  and  on  the  other  hand  deprived  synapses  fluctuate 
randomly  in  a  BD  situation.  It  is  shown  that  the  efficacy  of 
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deprived  synapses  decreases  exponentially  if  the  neuron  is  highly 


selective  over  the  environment  of  the  opposite  eye.  Hence  it  is 


predicted  that  in  an  MD  environment .  selectivity  development  in  the 


open  eye  must  precede  loss  of  responsiveness  in  the  closed  eye 


(Figure  14). 


Consider  a  neural  state  (m*,m*,q).  where  mj  and  m£  are 


synaptic  vectors  corresponding  to  left-eye  and  right-eye  inputs 


respectively,  which  is  driven  by  patterned  input  to  the  left  eye 


(d^(t)  =  d(w(t))  periodic  over  w)  and  pure  noise  to  the  right  eye 


( d r ( t )  =  n(t)  uniformly  distributed  about  0).  In  general,  the 
state  m*  fluctuates  randomly  and  m*  is  driven  to  a  high- 
selectivity  equilibrium  state  (m*,m*,q)  where  (20)  is  satisfied. 


<4i(m*-dJl,q*)  d(w)>u  -  0 


The  expectation  values  of  m^  and  mr  over  the  entire 


environment  D(u.n)  can  be  calculated: 


<m "  <$(«*»$•  d(w)  +  m**n,q*)d(u)>w^n 

■  0  +  <4'1(m*.d(w)  ,q*)  [m*>  n]d((u)^u  n 


<4r>.,. 


■  ($(mj*d(u>)  +  m*»n,q*) d(u>))^  n 

-  (t1(m*.d(u))Iq*)(m*.nln>w  n 

-  <^1(m*-d(w) ,q*)>w  <(m*.n)n)n 


2 


Initial 


Ei&ure  IAjl  Monocular  deprivation  sequence.  Computer  simulation 
illustrates  the  model’s  prediction  that  the  neuron  must  first 
become  highly  selective  to  patterns  from  the  open  eye  before 
that  eye  dominates  the  neuron. 


35 


The  expectation  value  < (mr* n)n>n  is  parallel  to  mr.  For 
an  N-cube  with  each  component  n^  varying  between  -a  and  +as 

Z_i _[+•  [+' 

(2a)”  I  •"/  Vj”l  *V"'lnN 

J  J  -&  J  -a 


Consider  a  constituent  integral  over  n^: 

f+a  r 
.a  “jnjni 


dn. 


2amTn,n 


J  ji 

0 


2  3  r 

Tmi 


if  ij*k  and  jj*k 
if  i»k  or  j«k  but  not  both 
if  i  ■  J  -  k 


Hence: 


<<mr  n)ni>n 


(2a) 


/-»  \N-1  2  3  r  a  r 
-  (2a)  3®  “i  "  ^  (23) 


A  similar  calculation  is  done  for  n  distributed  on  an  N-sphere: 


'  ,,»rl"t|n»rP7  n  -  <"«)  n  "r 


(24) 


By  spherical  symmetry: 


(nl)  „  ■  £<"•">„  ■?  vxSTiy  jv  dN" 

(»,a)  ). 


1 

N  V(N, 


x2  dV 


Since  V(N,x)  =  VNxN,  dV  =  NVNxN_1dx 


/  2v  1  fS  „  N+l  .  .  1  V 

<nm)n  "  V(N,a)  jn  V  dx 


N+2 


N 


VMa  N+2 

N 


2 

a 

N+2 


(m  •  n)  n  „ 
r  n 


N+2  mr 


(25) 


Thus  by  (22)  one  sees  that  <mr>„,#n  is  parallel  to  mr  and 
that  if  is  negative*  then  mr  decays  exponentially.  Since 

♦l  is  negative  when  m^.d^(«u)  is  near  zero*  which  is  true  over 
most  of  the  environment  when  m^  is  a  highly  selective  state*  such  a 
state  should  satisfy  <0.  Low  selectivity  states  are  thus 

unstable  in  two  senses:  (1)  m^  is  unstable  with  respect  to  dfc  and 
(2)  <$£>  may  be  positive  thereby  driving  mr  to  higher  magnitude. 


Selectivity  Minimization 

The  functions  $  and  <j/  satisfying  (26)  drive  the  model  neuron  (5) 
to  minimum  selectivity. 

sgn(*s)  -  -  sgn(x)  sgn(x  -  qp) 

(26) 

Yg  =  B(x-q)  ;  B  >  0 

where  0  <  p  <  1 

Note  <J»g=  ^  and  that  (11)»  (12)  and  (26)  can  be  reformulated  into  a 
single  model  (27) . 


sgn(^FP(x,q)) 

♦^Cx.q)  5 


-  a  Bgn(x)  sgn(x  -  qp) 
6(x-q)  ;  B  >  0 


(27) 


For  a>0  and  p>l*  the  cell  seeks  maximum  selectivity;  for  «<0  and 


0<p<l.  the  cell  seeks  minimum  selectivity;  otherwise*  the  system 
diverges. 


1 hfi.  li±ll  Dimensional  System 

Again  consider  a  one-synapse  cell  driven  by  a  steady*  clean 
signal.  As  in  the  case  of  the  D-cell*  the  S-cell  equation  (26) 
reduces  to  a  deterministic*  autonomous  system  (28).  All  trajectories 
converge  to  (1*1)  unless  (0*0)  is  the  initial  point. 


®(t>  -  $g(m(t) ,q(t)) 

q(t)  -  6  (m(t),q(t)> 


(28) 


The  origin  is  unstable  for  all  and  there  are  no  limit  cycles 
(Theorem  2) . 

Theorem  2*.  With  the  exception  of  (m(0)*q(0))  =  (0.0)*  all 
trajectories  driven  by  (28)  go  to  (m(t )  ,q(t )  )-+(l *1 )  as  t-*">. 

Proof.  As  in  theorem  1*  consider  four  regions  (Figure  15). 


(sgn  m.  sgn  $)  * 


(+1.  +1) 
(-1.  +1) 
(-1.  -1) 
(+1.  -1) 


•  Ri 


:  R2 

•  R-j 


The  point  is  monotonically  driven  to  satisfy  the  simultaneous 


38 


conditions  m=0,  q=0  and  it  is  always  driven  away  from  the  axes  m=0 
and  q=0.  (QED) 


Higher  Dimensionalities 

A  two  synapse  cell  in  a  two-pattern  (independent)  environment  is 
driven  to  (mj.n^.q)  =  (1,1,1).  a  zero  selectivity  state,  with 
the  exception  (in  a  noiseless  environment)  of  the  initial  point 
(m(0), n^O) ,q(0))  =  (O.O.qQ).  The  other  state  of  zero- 
selectivity,  namely  the  origin,  is  always  unstable.  The  highly 
stable  asymptotic  behavior  of  the  (1+1) -dimensional  system  is 
maintained  in  (2+1)  dimensions  (Figure  16). 

Due  to  the  parallel  aspect  of  the  modification,  patterns  driving 
the  cell  below  qP  are  "encouraged**  and  patterns  yielding  x  >  qP 
are  "discouraged**  (Figure  17)#  This,  combined  with  the  influence  of 
q,  causes  the  responses  to  the  various  stimuli  to  converge  toward  a 
common  value.  This  can  be  achieved  for  6ets  of  independent  patterns 
(Figure  18a).  For  dependent  pattern  sets  it  may  be  possible,  but  not 
in  general  (Figure  18b).  Thus,  qualitatively,  in  the  sense  of  Gati 
and  Tversky  (1982),  the  synapse  vector  seeks  the  pattern  component 
most  common  in  its  environment. 


W  U  A  3  i 


Figure  1&*  Projection  fii.  IM  (2+1)  fizsfcll  into  Hl£  m- plane.  As  in 
Figure  9>  q  is  idealized  to  be  the  expected  value  of  m*d  over 
the  orthogonal  environment  ((1*0)  »  (0.1)).  In  this  figure 
p  =  0.5  and  both  nullclines  (parabolas)  are  attractors. 
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X 
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0  i - 

(a) 


0  i - 

(b) 


ZiiHLS.  JiLa.  Selectivity  minimization,  (a)  An  environment  of  7 
independent  patterns,  (b)  50  patterns  in  7  dimensions, 
(c)  Noncircular  50  pattern  environment  in  7  dimensions. 
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B*  Aa  iM+ll  -  parallel  Model. 


In  this  section,  an  (N+l)-parallel  model  is  presented  (29). 
Again  $(x,q)  is  bounded  and  continuous.  The  function  6(q)  gives  the 
value  of  x  for  which  the  signs  of  $  and  change. 

sgn($)  -  a  sgn(x)  sgn(x  -  6(q)) 

(29) 

t|i  -  a  B  $(x,q)  x 
a  -  ±1  ;  B  >  0 


It  need  not  be  specified  as  a  power  function  as  was  done  in  the  FP 
model,  however  it  is  subject  to  certain  restrictions.  The  a 
parameter  determines  whether  the  cell  generalizes  (a=-l)  or  seeks 
maximum  selectivity  (a=+l).  Not  only  is  this  model  parallel  in  the 
sense  of  (5)  (aid),  but  the  change  in  the  neuronal  state  (m.q)  is 
parallel  to  the  (N+l)  -  activity  vector  (ad.Bx).  Thus  the  system 
can  be  expressed  more  concisely  in  terms  of  an  "(N+l)  formalism" 
(30).  The  model  is  given  in  a  general  form  in  that  e(q)  is  not 
specified,  so  that  overall  properties  of  the  system  (30)  can  be 
examined.  A  more  specific  example  is  presented  in  Part  IV. 


where  sgn(«(x,q))  -  sgn(x)  sgn(x  -  0(q)) 

o  -  ±1  ;  B  >  0 

0(q)  nondecreasing  &  continuous 


AO 


(l+i)  -  Oineaaiaaa 


The  one  synapse  -  one  pattern  situation  is  examined 
Synapse  m  receives  a  noncorrupted  steady  signal  d=l,  and 
giving  (31). 


first . 
so  x  =m 


(31) 


D-cell 
'cell 


o  -  i  +1  : 

l-l  :  S-c 

The  trajectories  in  the  m-q  plane  are  seen  to  be  parabolas  (32) 
since  (31)  gives  the  path  derivative  ^  ■  aBm  (Figure  19). 


_  _  „  ,  a6y  2  2. 

4  *  90  +  2~(«n  -  tDq) 


(32) 


For  the  S-cell,  the  conditions  on  6(q)  are  that  it  be 
continuous,  nondecreasing  and  positive  on  some  domain  [qfl,qj)]. 
If  qfl>0,  then  6(qa)=0  and  *(x,qa)>0  for  all  x.  Remember  that 
xCtO,").  If  q^  is  finite  then  6( )  is  not  and  *(x,  q>qb)<0 
for  all  x. 


The  D-cell  needs  additional  restrictions  on  9(q)  for  global 
stability  of  (31).  If  q(t=0)  is  very  large,  the  synapses  will  "die** 
-  i.e.,  (m,q)  converges  to  a  point  on  the  q-axis.  This  is  not  a 
serious  problem.  If  necessary,  restrictions  can  be  put  on  the 
initial  state.  Possible  divergence  of  the  system  is  more 
problematic.  If  <Kq)  does  not  increase  sufficiently  fast  as  q  gets 
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large,  some  trajectories  (m(t)tq(t))  will  grow  without  bound. 
Therefore,  we  impose  (33)  on  6(q).  Note  that  there  are  many 
functions  which  can  be  used  for  both  cell  types. 

'  0  <33) 

Unlike  the  FP  model,  this  system  does  not  have  a  finite  number 
of  isolated  equilibrium  points.  Instead,  the  equilibrium  points 
define  a  locus  consisting  of  the  q-axis  (m=0)  and  the  curve  m=e(q). 
All  points  satisfying  m=0(q)  are  the  attractors  with  the  possible 
exception  of  (0,qa).  Points  on  the  q-axis  can  be  either  stable 
(attractors)  or  unstable  equilibria.  For  aq>aqa  they  are  stable, 
otherwise  they  ""repel"  states. 

Stability  in  noisy  environments 

Isolated  attractors  are  generally  stable  in  environments 
corrupted  by  a  not-too-large  noise  level.  However,  continuous 
equilibrium  loci  are  subject  to  noi  e-driven  ""creep"*  effects. 

Chaotic  perturbations  may  preferentially  drive  the  neuronal  state 
along  the  equilibrium  surface.  Consider  a  one-synapse  cell 
receiving  a  signal  d(t)  =  1+  e(t)  where  e(t)  is  a  stochastic  variable 
uniform  on  C-a,a]  where  a<<l.  The  stability  of  the  system  (34)  can 
be  analyzed  by  expanding  about  an  equilibrium  point  (mQ,qg) 
where  mQ  =  0(q). 


I 


1 


Figure  19.  Parabolic  trajectories.  The  (1+1)  neuron  follows  a 
parabolic  path  in  the  m-q  plane  for  any  initial  point.  The 
function  e(q)  determines  at  what  point  along  the  trajectory  the 
path  direction  changes.  Here  0(q)  =  q^.  (a)  D-cell. 

(b)  S-cell. 


i- 


i 


t(rarfme(t)  ,q) 


a  +  ac 


(34) 


0m  +  Bmc 


) 


The  effect  of  noise  on  the  stability  of  q-axis  equilibrium 
points  is  not  relevant  to  this  thesis.  The  expected  value  of  (m,  q) 
is  calculated  over  e  in  (35).  For  either  value  of  a,  the  expected 
perturbation  direction  i&  along  £  trajectory  and  hence  the  neuronal 
state  tends  to  return  to  (niQ,  qp). 


(35) 


Hill  z  Dimensions. 

The  selectivity  properties  of  this  (N+l)-parallel  model  for  a 
two-synapse  cell  in  a  two-pattern  environment  {d* ,d2  }  resemble 
those  of  the  FP  model.  However,  we  again  have  a  locus  of  connected* 
rather  than  isolated*  equilibrium  points.  These  points  lie  along 
four  curves  intersecting  at  the  point  (mltm2,q)  =  (0.0*qa). 

The  four  curves  correspond  to  the  following  intersections  of 
surfaces  (Figure  20): 


fj&UHe.  20.  (2+1)  -  space.  The  four  equilibrium  curves  and  the 

four  regions  defined  by  the  surfaces  m*d  =  0(q)  and  m-d  =  0 
dC{dl,d2}  are  shown  for  0(q)  =  q2  in  an  orthogonal 
environment . 
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Cq  :  m.d*  =  m*d2  =  0  (the  q-axis) 

Cj  i  m*d^  =  6(q)  i  m*d2  =  0  | 

>  (maximum  selectivity) 

Cj  :  m*d*  =  0  ;  m*d2  =  0(q)  j 

Cj2t  m*d^  =  m*d2  =  6(q)  (minimum  selectivity) 

The  surfaces  partition  the  space  into  four  regions,  according  to 
the  sign  of  $  resulting  from  each  of  the  two  patterns.  For 
m*d^  >  9(q)  and  m*d2  >  0(q),  q  increases  until  the  one  or  both 
patterns  cannot  stimulate  the  neuron  above  6(q).  Neuronal  states  in 
the  opposite  region  (m*d  <  6(q)  for  both  patterns)  are  likewise 
forced  out  of  that  region.  Therefore  the  state  generally  finds 
itself  in  a  region  “between"  the  surfaces  m’d^  =  8(q)  and 
m*d2  =  0(q)  or.  in  the  case  of  the  D-cell,  possibly  on 

the  q-axis  (if  q(t=0)  is  “too  large"). 

Once  the  trajectory  is  between  the  two  surfaces,  that  is  for 
m*dl  <  0(q)  <  m*dJ  where  i  f  j,  its  behavior  depends  on  a.  For 
a=+l,  the  trajectory  converges  to  some  point  on  Cj  and  hence  the 
neuron  becomes  maximally  selective  whereas  for  a=-l  the  trajectory 
goes  to  a  minimally  selective  point  on  C^. 


HighfiX  Dimensionalities 

A  linearly  independent  environment  of  N  patterns  admits  2** 
equilibrium  curves  in  the  (N+l)  -  state  space  of  the  neuron.  These 
curves  lie  at  the  intersections  of  the  surfaces  m*dL  *  0(q)  and 
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ra.d1  =  0  for  i  =1*...*N.  Of  these*  two  (one  trivial  and  one  non¬ 
trivial)  represent  states  of  minimum  selectivity  and  N  represent 
states  of  maximum  selectivity  —  one  per  pattern. 

The  behavior  of  the  model  in  more  complex  environments  is 
difficult  to  analyze.  Numerical  techniques  suggest  that  for 
circular  environments*  D-cells  seek  maximum  selectivity  and  S-cells 
seek  minimum  selectivity.  Toroidal  environments  have  also  been 
simulated  for  which  the  pattern  depends  on  two  stochastic  parameters 
and  Computer  results  indicate  that  a  D-cell  tunes  with 

respect  to  one  parameter  only  (see  Figure  13)  and  that  S-cells  seek  a 
uniform  response  across  the  entire  environment. 
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IIIul  Construction  d  aiL  (N+l)-parallal  Model  using 
Antagonistic  Mechanisms 

A  function  satisfying  the  conditions  on  4  in  (30)  can  be 
constructed  (36)  as  a  q-coupled  difference  of  tvo  non-negative 
monotone  increasing  saturating  functions  satisfying  certain  criteria 
(Figure  21). 

♦(x,q)  =  o2(x)  -  qOjU)  (36) 


The  subtractive  nature  of  4  suggests  an  antagonism  between  two 
synaptic  modification  mechanisms  -  one  potentiating  [strengthening) » 
and  the  other  depotentiating.  It  is  seen  that  as  q  varies.  o2  and 
qoj,  intersect  at  different  values  x=e(q). 

There  are  two  conditions  that  both  Oj  and  o2  must  satisfy  to 
let  $(x.q)  fulfill  the  conditions  of  the  (N+l)  parallel  model: 

First.  Oj(0)  =  o2(0)  =  0  and  second,  the  sign  of  the  Wronskian 
8gn(W(o j(x).o2(x)))  =  a  for  x>0  (Theorem  3). 

Theorem  2»  If  sgn(W(aj(x),  o2(x)))  =  a  for  x>0  and 
°l(0)  =  o2(0)  =  0.  then  sgn(t(x.q))  =  a  sgn(x)  sgn(x-6(q)). 

Proof.  The  proof  is  as  follows:  First,  since  4(0*q)  =  0  for 
all  q.  °j(0)  =  °2(0)  =  0  must  be  satisfied.  The  other  sero  of 
♦(x.q)  is  at  x=  6(q).  or  q  sB'^x)  =  o2(x)/oj(x).  The  slope  of 
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6  with  respect  to  x  is  evaluated  (37)  at  this  point  and  shown  to  be 
of  the  sane  sign  as  the  Wronskian  WCoj.Oj)* 


oAx) 

'  °2<x)  -  ^00  °i<X> 


1 

C^(x) 

1 

0^(x) 


[ol°2  "  0i°21 

a^x)  a ^  (x) 

o*(x>  cr^x) 


(37) 


Therefore  if  sgn(W(o  ^ ,o 2))  is  positive  or  negative  definite  for 
x>0,  so  is  *x(x=6(q))  and  ♦  =  (jj'Vi  has  at  most  two  zeros:  x=0 
and  x=6(q).  D-cells  and  S-cells  can  be  characterized  by 
sgn(<tx(x=0(q) ))  as  well  as  by  o  since  a  ~  Bgn  (*x(x=e(q) )) ,  hence 

a=sgn(W(aiKJ2^* 

q  as  a  time  average 

As  in  (13)»  the  quantity  q(t)  is  a  kind  of  tine  average  of  the 
the  post-synaptic  activity.  Equation  (38)  can  be  solved  for  q(t) 
using  an  integrating  factor  to  give  the  solution  (39). 


(38) 


q(t)  -  8<«2(x)  -  qo^x))  x 


q(t)  -  qQ  exphfi/J  x(t")a1(x(t"))  dt"  ) 

09) 

+  fij  expl-p/g  x(tH)o1(x(t,,))J  o2(x(t'))  x(t')  dt' 

Note  that  the  first  term  vanishes  as  t4*.  Again*  q(t)  is  seen 
to  be  an  exponentially  damped  average  of  some  measure  of  the  cell's 
activity  (in  this  case  xo2(x)). 


Some  allowed  functions  g.(z) 

For  simplicity's  sake*  only  functions  and  o2»  which  are 
identical  in  form  while  having  different  horizontal  and  vertical 
scales,  are  considered  (40). 


‘“,1W  2 

Vni  >  0 

(40) 

where  c(0)  -  0 

o(l)  -  \ 

(x-ni  gives  y-max 

of 

11m  o(x)  ■  1 
**• 

(u  Is  max(o^)) 

a '  (x)  >  0 

for  all  x 

Since  the  product  Ujj^  can  factored  from  the  Vronskian.  W  ia 
independent  of  Mj  and  Thus,  consider  the  determinant 

JC  X 

W(o(— ),o(-— )).  The  following  Theorem  (4)  establishes  a  sufficient 
°1  n2 

condition  on  o  so  that  t(x»q)  will  have  at  most  one  zero  for  x>0. 

Theorem  4*.  If  o(x)  satisfies  x(o*)2  -  xoo"  -  oo'  >  0.  then 
for  a  given  value  of  q.  there  will  exist  either  one  or  zero  values 
x>0  for  which  ♦(x.q)  =  0.  depending  on  the  value  of  q. 

Proof .  W(o(-£-),o(~))  is  evaluated  and  expressed  so  that  its 
nl  n2 

sign  depends  upon  a  difference  of  two  terms,  separated  according  to 

»  (*1). 


W<o(J  ))  -  i0’(i)0(J) 

^1  ^2  ^2  ^2  ^1 


-  )o(i  ) 

nl  n2 


o’Cf  ) 

n2 

n«o  (~  ) 
2  n2 


-  io’(-)o(-) 

nl  V  V 


o*(J  ) 

nl 


nl0(n  > 
1  nl 


(41) 


Hence,  the  condition  (42)  is  sufficient  to  our  purpose  and  we 
see  that  ago  (♦x(8(q),q))  depends  on  the  agn  (n2”ni)* 
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d_ 

dn 


i 

no<*> 


for  all  x>0  ,  n>0 


(42) 


The  derivative  with  respect  to  r\  is  evaluated  and  the  substitution 
8  =  —  completes  the  proof  (43). 


d_ 

dn 


(no(*))2 

-8o(a)o"(s)  +  8(o'(8))2  -  g,(s)a(8) 
(no(s))2 


(43) 


2 

*>  x(o')  -  xoo"  -  o'c  >  0  is  sufficient!  QED 

Under  this  formulation  the  feature  abstracting  quality  (S  or  D) 
of  a  neuron  depends  on  which  of  two  saturating  functions  increases 
more  rapidly  rather  than  on  the  sign  of  a  variable  in  the  model's 
differential  equation.  A  mechanistic  interpretation  poses  two 
biochemical  processes  as  antagonists  modulating  synaptic  plasticity* 
each  increasing  with  the  activity  of  the  postsynaptic  cell.  The  S/D- 
nature  of  the  cell  then  depends  on  certain  details  of  the  functional 
relationships  relating  each  process  to  the  cell's  activity. 


Among  functions  satisfying  the  condition  (43)  are  those 


sigmoidal  functions  belonging  to  the  class  (44) 
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o(x) 


xp  +  1 


p  >  1 


(44) 


The  parameter  p  determines  the  "sigmoidocity  of  the  curve.  Near 
x=0,  the  function  behaves  like  the  power  function  xP.  This 
suggests  that  one  or  both  of  the  functions  cr^  may  be  linked  to  the 
firing  rate  of  the  cell*  which  is  seen  (Chapman,  1966)  to  be  an 
increasing  bounded  function  of  the  summed  potential  with  a  threshold. 

This  system  is  no  longer  precisely  of  the  form  (30).  While  the 

properties  of  the  D-cell  model  are  maintained,  the  S-cell  version  of 

this  variation  is  somewhat  different.  Here,  (-—)  and  <i>  are  always 

di  x 

the  same  sign:  that  is,  there  is  no  a.  Thus,  the  system  is  written 
in  a  more  appropriate  form  (45),  where  o  satisfies  (43).  For  a  D- 
cell,  n2>ni  whereas  the  opposite  holds  for  an  S-cell. 

(?)  ■  °<?2>  -  (?)  <«> 
Examination  ol  the  H=1  ca&e 

For  N=l,  the  system  is  always  bounded.  As  in  (30),  the  synaptic 
weights  may  vanish  altogether  if  the  initial  value  of  q  is  too 
large.  Sample  trajectories  are  shown  (Figure  22)  to  converge  to 
points  on  either  the  nullcline  or  on  the  q-axis  above  the  nullcline 


locus  which  depends  on  rjj  and  ^  •  Note  that,  in  contrast  to 
Figure  19,  the  trajectories  for  the  two  cell  types  are  the  sane  in 
both  cases  and  the  nullcline  is  not. 
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The  surfaces  02(m*d1)  ~  qo^(m*dL)  =  0  for  i  =  1,2  are 
sheets  defined  by  a  q-translation  of  the  nullcline  above.  The 
pairwise  intersections  of  these  nullclines  and  the  planes  nud1  =  0 
with  each  other  consitute  the  locus  of  points  in  equilibrium  with  the 
environment  {d^-.d^}.  Whether  the  function  x  =  6“*(q)  = 

i®  monotone  increasing  or  monotone  decreasing  is  seen 
to  determine  the  feature  abstracting  properties  of  the  neuron. 

Figure  23  illustrates  the  (2+1)  dimensional  case.  For  0(q) 
between  m’d*  and  m*d^  the  neuronal  state  is  driven  toward 
maximum  selectivity  for  h2>r,i  and  toward  minimum  selectivity  for 
n2<h1 .  For  fixed,  as  nj increases  from  less  than  02  to 
greater  than  Oj*  the  surfaces  “pass  through”  one  another,  thereby 
reversing  the  directions  of  the  trajectories  in  the  m-pro jection.  At 
nj=n2  the  surfaces  coincide  and  the  neuron  seeks  neither  maximum 
nor  minimum  selectivity. 


ik 
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N>2 


In  higher  dimensional  systems*  computer  simulation  indicates 
that  for  n2>rli*  the  model  neuron  always  seeks  maximum  selectivity 
and  for  it  seeks  minimum  selectivity.  Simulation  of 

binocular  stimulus  environments  to  D-cells  give  results  consistent 
with  the  experimental  data  reviewed  in  Part  Ic. 
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IV.  An  Ensemble  Method  £sn.  Elec trophy Biological  Testing  d  Svnaptic 

Modification  Hypotheses 


Aj.  Asymptotic  Stability  and  Investigation  of  Equilibrium  St.atfiS 

Both  experimental  and  theoretical  considerations  allow 
elimination  of  several  hypothetical  excitatory  synaptic  modification 
rule  formulations.  In  particular  a  good  formulation  should  (1) 
allow  for  both  strengthening  and  weakening  of  a  synapse,  (2)  not 
allow  synaptic  strength  to  grow  without  bound,  and  (3)  not  allow  the 
sign  of  a  synaptic  strength  to  change  (a  synapse  must  be  always 
excitatory  or  always  inhibitory). 

Properties  (2)  and  (3)  are  particularly  relevant  to  the  present 
note.  Experiments  show  that,  with  repetition,  conditioning 
stimulation  that  initially  produces  increased  synaptic  strength  is 
eventually  ineffective  for  producing  change.  That  is,  the  continued 
stimulation  drives  the  synapse  to  an  equilibrium  state  (the  approach 
to  equilibrium  may  be  asymptotic).  Likewise  experimental  conditions 
which  initially  depress  synaptic  function  are  eventually  ineffective 
and,  particularly  from  a  theoretical  viewpoint,  provide  no  reason  to 
posit  synaptic  influence  reversing  modes  from  excitatory  to 
inhibitory  or  vice-versa.  Of  course  formulations  that  predict 
asymptotic  neural  states  that  change  due  to  shifts  in  environmental 
statistics  are  desirable. 
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Based  on  these  considerations,  we  now  restrict  our  concern  to 
formulations  that  yield  this  asymptotic  stability.  In  fact  this 
criterion  is  the  basis  for  the  proposed  experiments  and  yields  a 
method  that  is  insensitive  to  many  of  the  problems  of  other  classes 
of  laboratory  paradigms.  By  performing  all  experiments  after  an 
initial  asymptote  has  been  reached*  actual  values  of  postsynaptic 
excitation  and  presynaptic  frequency  can  be  ignored. 

Experimental  data  (see  section  IVe)  have  revealed  that  the 
strength  m  of  a  synapse  can  be  increased  or  decreased  depending*  at 
least  in  part,  on  the  presynaptic  activity  x  and  the  postsynaptic 
activity  y.  As  a  plastic  synapse  adjusts  to  a  constant  activity 
pattern  (xg.yg)*  in  its  pre-  and  postsynaptic  elements,  its 
efficacy  levels  off  (m-*0,  where  the  dot  indicates  derivative  with 
respect  to  time).  This  equilibrium  state  can  be  perturbed,  shifting 
m  to  either  positive  or  negative  values,  by  altering  the  constant 
activity  pattern  by  an  amount  (dx.dy)  to  (xj,yj).  The  dependence 
of  the  sign  of  m  on  the  signs  of  dx  and  dy  reveals  primary 
information  regarding  the  form  of  synaptic  modification  rule  (46)  and 
secondary  clues  to  plasticity  mechanisms. 


m 


m(x,  y,  .  .  .) 


(46) 


The  condition  m  =  0  thus  implies  that  some  quantities  varying 
with  x  and/or  y  have  achieved  a  static  equilibrium  which  can  be 
disrupted  by  perturbing  the  activity  pattern  (x,y).  Thus  by 
appropriately  adjusting  (x.y),  one  ought  to  be  able  to  induce 
potentiation  (m  >  0)  or  depression  (m  <  0) . 

By  investigating  the  dependence  of  the  sign  of  m  on  the  various 
manipulations  (dx,dy),  one  can  determine  the  approximate  form  of  the 
modification  function  (46)  in  the  neighborhood  of  ( *q  » yg ) •  Let  a 
given  experiment  be  characterized  by  (sgn  dx»  sgn  dy)  where  each 
component  is  a  member  of  the  set  ( +,  0  ,  ♦)  and  let  each  possible 
result  be  written  as  (sgn  dx,  sgn  dy;  sgn  dm).  For  example,  the 
paradigm  x^  >  xq  and  yj  =  yg  is  expressed  as  ( +,0)  and  if  the 
result  is  m  <  0,  then  we  would  write  (+.0  ;  -). 

There  are  nine  possible  experiments  of  this  type.  The  (0,0) 
experiment  is  assumed  to  have  a  null  result,  so  considering  3 
possible  results  for  each  of  the  remaining  eight  paradigms  gives 
3®  or  6561  combinations  of  possible  outcomes.  Certain  (seemingly 
inconsistent)  combinations  imply  very  complex  modification 
functions.  We  will  confine  our  analysis  to  predictions  of  some 
simpler  theories. 

Consider  a  set  of  (bi) linear  approximations  (47)  to  m  in  the 
neighborhood  of  an  equilibrium  state  in  which  a  constant  activity 
pattern  (xg»yQ)  has  driven  the  synaptic  efficacy  to  a  value  m*. 
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m  -  (xQ,y0,m*)  (x 

m  -  K^(x0,y0,m*)  (y 

.  (e)  * 

m  -  Kv  ;  (x0,yQ,Tn  )  (x 


V 

(47a) 

V 

(47b) 

V  <y  •  v 

(47c) 

The  equilibria  in  these  hypothetical  systems  result  from  a 
balance  in  (a)  presynaptic,  (b)  postsynaptic.  or  (c)  both  pre-  and 
postsynaptic  (independently)  terms.  In  each  formula*  K  is  positive 
(or  negative)  semi-definite  and  considered  constant  in  these 
approximations.  The  zeros*  or  balance  points.  bx  and  by  vary 
according  to  their  own  rules  following  *  tine  SQDX&e  assumed  Jt£  lie 
fast  enough  that  we  can  reach  equilibrium  in  the  laboratory  and 
slow  enough,  that  we  can  measure  the  immediate  change  in  m 
resulting  from  the  perturbation. 

These  formulae  represent  classes  of  theories  for  various  neural 
systems  including  probability  learning  (47a)  (e.g.  Levy  &  Desmond* 
1982).  development  of  specificity  in  visual  cortex  (47b)  (Bienenstock 
et  al*  1982)*  and  associative  memory  (47c)  (Kohonen*  1977;  Sejnowaki* 
1977a*b).  The  predictions  of  the  paradigm  for  each  of  these  models 
are  given  in  Table  1 . 


Su.  Extension  Ifl.  Populat ion  Studies 

The  above  analysis  for  individual  synapses  can  be  extended  to 
population  stimulus-response  paradigms.  Instead  of  measuring  the 
effect  of  a  single  afferent  upon  a  neuron*  consider  the  macroscopic 
effects  of  a  stimulus  pattern  over  an  axon  bundle  on  the  target 
population.  Just  as  the  response  of  a  single  cell  to  a  test  pulse 
in  an  afferent  axon  is  a  measure  of  synaptic  efficacy*  the  net 
response  potential  of  a  population  to  a  test  pulse  pattern  is 
similarly  a  measure  of  the  net  efficacy  of  the  synapse  ensemble 
corresponding  to  the  afferent  bundle. 

As  all  neurons  in  the  postsynaptic  population  approach  an 
equilibrium  steady  state  in  response  to  a  constant  stimulus  pattern* 
so  must  the  synapse  ensemble  efficacy  (SEE).  The  relationship 
between  the  SEE  and  the  individual  synaptic  strengths  is  closely 
related  to  the  concepts  of  microstate  and  macrostate  introduced 
by  Amari  (1974)  in  his  method  of  statistical  neurodynamics.  It  will 
be  noted  that  Amari  is  concerned  with  activity  levels  rather  than 
synaptic  states*  hence  one  should  be  aware  that  the  test  stimulus 
evokes  an  activity  level  that  reflects  the  synaptic  state  (either  of 
a  single  synapse  or  a  synapse  ensemble).  The  SEE  concept  can  be 
generalized  to  a  vector  or  matrix  quantity^  describing  the  pattern 

^  The  terms  "vector'*  and  "matrix"  are  meant  in  the  multi¬ 
dimensional  sense.  Tensor ial  mappings  between  coordinate 
systems  do  not  hold  in  general. 
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of  ensemble  efficacies  of  several  bundles  to  a  population  or  to 
several  populations. 

Consider  the  special  case  in  which  several  bundles 
(Sq.Sj,...  SN)  project  to  a  common  population  R,  one  projection, 

Sq,  being  much  denser  than  the  others  (Figure  24).  That  is  the 
activity  in  Sq  can  completely  dominate  the  response  in  the  target 
population  thereby  providing  a  convenient  mechanism  for  experimental 
control  of  postsynaptic  activity.  Thus  the  activities  both  pre-  and 
post  synaptic  to  the  synapses  from  the  sparse  projections  can  be 
imposed  by  the  experimenter.  However,  the  dense  or  modulatory 
projection  modifies  passively,  that  is  the  activity  postsynaptic  to 
this  projection  is  principally  dependent  on  the  projection's 
(presynaptic)  signal.  The  synapses  from  the  sparse  projection 
undergo  active  modification  in  the  sense  that  they  are  driven  by 
activities  with  no  strong  causal  relation. 

Caution  must  be  exercised  with  respect  to  the  passively 
modifying  modulatory  projection.  Assuming  this  projection  is 
plastic,  one  must  recognize  that  a  constant  stimulus  to  this  bundle 
does  not  necessarily  elicit  a  constant  postsynaptic  response. 
Therefore  before  one  proceeds  with  an  equilibrium  perturbation 
paradigm  in  a  dense-sparse  system,  both  the  sparse  SEE  under 
examination  and  the  dense  SEE  must  achieve  sufficiently  stable 
activity  levels.  Presynaptic  perturbations,  that  is  perturbations  in 
the  stimuli  to  the  sparse  bundles,  can  be  used  to  provide 


figure  21m.  Zh£  affeteat.  vastox  £  projects  £&  a  population  £. 
(a)  The  population  R  is  the  common  target  of  several  axon 
bundles  S-.  The  projection  Sq  i6  much  denser  than  the 
others,  tb)  A  single  cell  in  R. 
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information  about  the  x  dependence  of  d  (the  middle  columns  in 
Table  1). 

The  investigation  of  postsynaptic  effects  is  not  so 
straightforward.  To  alter  the  steady-state  activity  in  R .  we  must 
change  the  stimulus  delivered  to  Sq,  thereby  forcing  a  change  in 
the  corresponding  SEE.  This  unfortunate  aspect  of  the  experimental 
design  is  not  fatal.  The  procedure's  validity  can  be  ensured  by 
monitoring  the  SEE  from  Sq.  As  long  as  the  change  in  the  efficacy 
of  the  Sq  ensemble  follows  the  change  in  the  Sq  stimulus*  it  is 
valid  to  infer  that  the  activity  in  the  target  population  is 
similarly  affected. 

However,  if  the  Sq-SEE  changes  in  the  direction  (+  or  -) 
opposite  to  the  perturbation  in  the  Sq  stimulus,  it  is  difficult 
or  impossible  to  ascertain  the  change  in  the  activity  in  R.  If  the 
connection  is  excitatory*  the  initial  change  in  the  Sq-SEE  will 
most  likely  follow  the  change  in  the  stimulus.  On  the  other  hand, 
as  is  seen  in  the  next  section,  certain  models  predict  that  the 
change  in  the  net  asymptotic  efficacy  value  opposes  the  change  in 
the  stimulus  after  initially  following  it. 


£jl  fflfljgive  Potentiation 


Investigation  of  passive  potentiation  is  not  only  a  necessary 
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control  stage  for  the  experimental  framework  we  have  outlined.  It 
may  also  be  used  as  a  self-contained  technique  for  distinguishing 
between  certain  associative  models.  As  a  control  for  active  synapse 
modification  experiments,  passive  potentiation  in  the  dense 
projection  Sq  must  be  understood  for  the  reasons  stated  in  the 
above  section. 

In  this  section  we  will  examine  the  utility  of  a  passive- 
potentiation  study  as  a  stand-alone  experiment  to  reveal  traits  of 
the  modification  process  that  are  important  from  a  mathematical 
standpoint.  Consider  two  models  both  having  postsynaptic 
equilibrium  points  y  =  by  as  in  (47b),  but  differing  in  their 
evolution  equations  for  the  balance  point  itself  (48).  The  time 
scale  for  the  modification  of  by  is  assumed  to  be  slower  than  that 
for  m.  Notice  that  equilibrium  can  be  attained  only  when  both  m  and 
by  vanish.^ 

b  «  B(y  -  f(y>bw)) 
y  " 

If  f(y»by)  =  by,  then  there  is  a  single  criterion  for 
equilibrium,  namely  when  y  =  by .  Otherwise  (as  in  the  FP  model), 
the  system  has  a  much  more  restricted  set  of  equilibrium  points.  In 
this  case  y  must  simultaneously  satisfy  two  conditions  and  so  in 
general  only  certain  isolated  values  of  y  are  allowed. 


The  expectation  values  of  m  and  b  over  their  pattern 
environments  should  be  zero  at  equilibrium.  Tn  the  experi¬ 
mental  paradigms  discussed  here,  only  one  pattern  is  presented 
for  the  duration  over  which  the  expectation  values  are 
integrated. 


61 


Consider  an  experiment  similar  to  that  described  in  the  laBt 
section!  in  this  case  for  the  measurement  of  the  SEE  of  a  dense 
(controlling)  projection.  As  before*  the  synapse  ensemble  is  driven 
to  asymptotic  efficacy.  Hypothetically*  (47b)  has  been  accepted  as  a 
description  of  m  local  to  equilibrium.  The  ne^t  step  is  to  determine 
whether  (48)  can  be  expressed  with  f(y,by)  =  by .  When  the  system 
settles  down*  the  constant  stimulus  to  the  axon  bundle  is  shifted  and 
again  held  constant.  If  the  condition  y  =  by  alone  is  sufficient 
for  equilibrium*  the  prediction  of  the  coupled  equations  (47b»48)  is 
that  the  SEE  will  monotonies lly  shift  to  a  new  asymptote. 

On  the  other  hand,  if  y  is  forced  to  the  same  equilibrium  value 
by  both  constant  stimulus  level'  then  the  SEE  must  initially  follow 
the  stimulus  change*  peak  at  an  extreme  (relative  minimum  or 
maximum),  and  asymptotically  seek  a  net  shift  opposite  to  the 
stimulus  shift.  To  appreciate  this,  consider  a  more  detailed 
example  (Figure  25).  The  synapse  ensemble  is  initially  potentiated 
to  a  steady  state  value  by  a  stimulus  of  intensity  Xj  over  a 
time  period  Tj ,  after  which  a  stronger  stimulus  X2  is  delivered 
for  a  period  T2  driving  the  SEE  to  a  new  asymptotic  value  M2. 
Following  (47b)  in  the  short  time  scale*  an  initial  synaptic 
increase  is  expected.  The  weak  criterion,  under  which  y  =  by  is 
sufficient  for  equilibrium,  allows  the  system  to  achieve  a  second 
steady  state  without  reversing  the  trend.  Otherwise  both  asymptotic 
population  responses  must  be  equal*  and  since  X2  >  Y-,  the  model 
predicts  M2  <  . 
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These  predicted  behaviors  of  the  SEE  result  from  the  interaction 

#  • 

between  the  coupled  differentiil  equations  for  m  and  b^.  This  is 
illustrated  by  the  global  state  diagrams  in  Parts  II  &  III  (Figures 
8  and  19).  Even  though  these  figures  represent  single  synapse 
cells,  the  SEE  of  a  multisynaptic  neuron  should  behave  accordingly. 

In  Figure  26,  we  see  that  a  shift  in  the  constant  stimulus  value 
causes  a  corresponding  shift  in  the  nullclines.  Thus  new  "rules’* 
are  established  in  the  state  space  and  the  synaptic  strengths  must 
obey  them.  Note  the  direct  approach  allowed  in  (a)  as  opposed  to  the 
spiral  trajectory  in  (b).  The  projections  of  these  paths  on  the  in¬ 
coordinate  give  the  Tj  time  courses  illustrated  in  Figure  25. 

IU  lesliag  Nonhebbian  Models 

Nonhebbian  (in  the  sense  that  they  are  not  multiplicative) 
models  are  not  described  by  any  of  the  formulations  in  (47),  hence 
they  predict  combinations  of  experimental  results  not  included  in 
Table  1.  These  theories  are  by  no  means  excluded  from  our 
experimental  approach.  The  only  requirement  is  that  one  be  able  to 
predict  qualitative  behavior  of  the  SEE  near  equilibrium  ior  various 
paradigms.  In  this  section  we  again  contrast  the  predictions  of  two 
variants  of  a  class  of  models. 


Recall  the  presynapt ical ly  balanced  modification  rule  (47a).  As 
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Figure  State  *p*ca  diagiaaa*  Two  cases  ere  illustrated:  (a) 
F(y»b  )  =  by  ,  and  (b)  other  Functions  £,  Dote  that  the 
m-projections  of  these  curves  give  the  predictions  for  M  in  the 
previous  figure. 


•  hebbian  rule*  Che  egression  for  »  should  be  faccorable  into  pre- 
and  post synaptic  components,  so  the  term  (x  -  bx)  should  be 
independent  of  postsynaptic  variables.  Consider  a  nonhebbian 
generalisation  of  (47a)  in  which  the  equilibrium  condition  is 
expressed  as  a  halanca  between  pre~  and  postsynaptic  activities 
(49). 

m  -  K(*0.y0,«*)  (*  -  i(y)bx)  (49) 

Prediction  combinations  for  various  functions  f(y) 
are  given  in  Table  2:  (a)  f(y)  inereasea  monotonically  I  (b)  f(y)  is 
constant  (hebbian  case)  s  (c)  f(y)  decreases  monotonically. 

Application  to.  JA&  BC-PC  Zattarai 

The  principal  input  to  the  hippocampus  arises  from  the 
entorhinal  cortex,  an  adjacent  structure.  Figure  27  gives  a 
schematic  view  of  the  bilateral  pathway  projecting  from  the 
entorhinal  cortex  (IC)  to  the  dentate  gyrus  (DG)  in  hippocampus . 
Plasticity  in  the  IC-DG  pathway  has  been  demonstrated  by  many 
investigators.  The  response  of  DC  granule  cell  populations  has  been 
shown  to  increase  as  a  result  of  strong  electrical  stimulation 
applied  directly  to  the  pathway  (Bliss  4  homo,  1973).  The  so-called 
long-term  potentiation  (LTP)  effect  has  been  shown  to  last  for  a 


DG 


left  right 


Fiffure  27.  The  bilateral  SC- DC  path mt.  This  diagram  is  based  OB 
the  data  of  Steward  &  V insect  (1978)  in  which  they  induced 
sprouting  in  the  crossed  pathway  and  found  all  new  crossed 
projections  to  be  collaterals  of  ipsilateral  projections.  The 
dotted  lines  indicate  the  sparse*  crossed  pathways.  Solid  lines 
are  the  denser  ipsilateral  projections. 
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period  of  months  (Douglas  6  Goddard.  1975).  Levy  6  Steward  (1979) 
have  simultaneously  stimulated  contra-  and  ipsilateral  pathways  for 
Hebb-like  modification.  The  above  development  of  the  8EE  approach  in 
this  section  was  motivated  by  this  system. 

In  the  proposed  experiment,  stimulating  electrodes  are  placed 
(see  Levy  &  Steward  (1979)  for  details  of  the  setup)  in  the  left  and 
right  angular  bundles  and  at  least  one  recording  electrode  is  placed 
in  the  dentate  gyrus,  either  unilaterally  or  bilaterally.  When  both 
stimulators  are  active,  the  DG  population  response  is  under  the 
nearly  complete  control  of  the  ipsilateral  electrode.  Thus,  the 
experiment  is  to  observe  the  change  in  the  contralateral  SEE 
(cSEE).  A  measure  of  the  cSEE  is  the  population  voltage  response  in 
DG  to  a  single  contralateral  pulse. 

One  must  be  careful  to  follow  the  cSEE  to  equilibrium  for  each 
driving  stimulus.  It  is  important  to  be  aware  of  the  possibility  of 
a  change  in  the  iSEE.  Bence  it  may  be  a  worthwhile  precaution  to 
measure  the  iSEE  as  well  as  the  cSEE  as  a  test  of  equilibrium. 
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A.  Xht  S-e«ll  JJL  A 


Bait 


Tonic  inhibition  can  easily  ba  shown  to  increase  specificity  in 
a  selective  neuron.  Experimental  evidence  for  this  phenomenon  is 
provided  by  8illito  (1975).  The  role  of  inhibitory  activity  in  a 
nodel  neuron  is  discussed  by  lienenstock  (1960).  In  that  paper  it 
is  shown  that  cells  lose  specificity  if  their  “ideal"  synaptic 
inputs  are  confined  to  positive  values.  Anatomical  data  does  not 
support  the  ootioa  of  the  ideal  synapse.  Feed-forward  inhibition  by 
an  S-cell  to  several  D*cells  all  having  roughly  the  sane  receptive 
field  supplies  than  with  a  generalised,  tonic,  negative  input. 


Consider  a  circuit  (Figure  26)  in  which  a  cortical  D-cell 
receives  strictly  excitatory  patterned  thalsmic  inputs  and  is 
inhibited  by  a  neighboring  cell  receiving  the  sane  input.  Assuming 
linear  response  properties,  the  D-cell  responds  according  to  (50). 
where  n  and  n  are  respectively  the  states  ef  the  D  and  1  (Inhibitory) 
cells  and  k  is  the  magnitude  of  the  inhibitory  1-D  coupling. 

x  «  n*d  -  k(a*d)  ■  (n  -  ka)*d  (50) 


Sy  restricting  a.  a.  and  k  to  be  noa-negative.  the  nodal  is  more 
consistent  with  contemporary  anatomical  theory.  For  n  and  k 


2A*.  Zb*  AA  A  ennttMt  nh««ie«r  Jtfl.  *  fc&ftll*  In  this 

staple  circuit,  a  generalising  unit  (S-cell)  inhibits  •  D-cell 
naif  only  over  their  colon  pattern  environment  thereby 
separating  the  patterns  for  the  D-cell. 


nooiodifublci  the  produce  kn  provides  an  arbitrary  offset  since 
d/dt(a-kn)  *  da/dt.  Provided  kn  is  sufficiently  Urge  for  all 
patterns*  there  exist  positive  synaptic  values  for  which  the  D- 
cell  has  aaxiaua  selectivity.  If  the  inhibitory  contribution  kn  is 
too  large  the  D-cell  will  never  fire.  Bence*  the  "allowed**  values  of 
kn  over  the  pattern  space  are  restricted  by  these  two 
considerations. 


Taking  a  aore  robust  approach*  let  the  inhibitory  neuron  seek 
ainiaua  selectivity  and  let  the  k-synapse  be  modifiable.  The  S-cell 
receives  only  excitatory  input  and  therefore  develops  according  to 
the  analysis  in  previous  sections.  Mote  the  response  x  of  the  D- 
cell  (50).  As  the  S-cell  approaches  equilibrium*  n  goes  to  sero  and 
d/dtfaj-kn^)  *  dmj/dt  -  n^dk/dt.  To  maintain  the  properties 
of  D-cell  parallel  modification*  k  should  modify  opposite  to  m 
(51)*  since  they  share  the  saae  post-synaptic  cell. 

k  -  -$(x,q)  •  [S-cell  output] 

(51) 

-  -♦(x,q)  •  (n*d) 


Hence  a  and  k  change  such  that  d/dt[(a-kn)'d]  increases  if  x>e(q)  for 


a  given  d*  and  di 


is  if  0<x<9(q).  This  circuit  will  tolerate  a 


auch  broader  range  of  initial  conditions  than  if  the  inhibitory  cell 
is  nonplastic.  The  tiae  course  of  both  S  and  D  response 
characteristics  for  a  (monocular)  circular  environment  is  given  in 
Appendix  C  using  the  TT  aodel. 


Note  that  a  tingle  S-cell  may  inhibit  several  D-cella  receiving 
inputs  from  the  same  receptive  field  (Figure  29)  »  each  tuning 
independently.  Recent  experimental  data  from  Hubei  and  Livingstone 
(1982)  and  Tootell  et  al  (1982)  support  this  kind  of  circuit.  They 
find  periodic  clusters  of  nonselective  cells  centered  in  ocular 
dominance  columns.  Also.  Hendrikson  et  al  (1981)  report  having  found 
the  GABA- synthesizing  enzyme  gluatamic  axid  decarboxylase  (GAD)  in 
localized  regions  arranged  in  a  strikingly  similar  pattern.  Their 
studies  in  monkey  visual  cortex  reveal  these  GAD-rich  areas  to  run 
parallel  to  the  centers  of  the  ocular  dominance  columns. 

This  approach  to  resolving  the  ideal  synapse  has  been  developed 
in  cooperation  with  Chris  Scofield  who  is  applying  it  to  a  theory  for 
the  development  of  the  topography  of  orientation  preference  across 
cortex. 

Conacata  Regarding  ihc.  Antagonistic  Mcchaaiiaa  Approach 

n  aa  a  lyaaptic  value 

As  is  mentioned  in  Fart  111*  either  of  the  saturating  functions 
o(x)  may  be  related  to  the  firing  response  of  the  neuron.  Consider 
the  term  qOj(x).  Since  q  modifies  in  a  hebb-like  fashion*  it  may 
reflect  the  efficacy  of  a  synapse.  Figure  30  depicts  q  as  the 
efficacy  of  an  autapae  (a  synaptic  junction  having  pre-  and 


Figure  2SU.  4a  8-cell  eubencee  eexecal  Ibcella*  a  single  s-celi 
can  provide  contrast  enhancing  feed-forward  inhibition  to 
several  D-cells. 


Figure  11L  Autmtm  mtuliLiail  o£  ♦.  In  this  circuit  the  neuron's 
output  is  fed  back  to  the  cell  as  the  function  o, .  This  nay 
be  accomplished  via  a  special  autaptic  connection  of  strength  q. 
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post synaptic  elements  that  are  processes  of  a  common  neuron  (van  der 
Loos  and  Glaser*  1972)). 

Whether  qoj(x)  inhibits  the  firing  of  the  cell  and  whether 
it  is  mediated  by  an  actual  autaptic  structure  are  details  not 
considered  here  except  for  the  following  observation.  If  an  explicit 
structure  mediates  the  subtractive  term  qoj(x),  such  structures  are 
expected  to  be  axosomatic  so  they  can  interact  with  the  quantity 
02(x).  (Note  that  x  is  a  somatic  quantity).  Calvet  &  Calvet 
(1979)  report  the  existence  of  axosomatic  autapses  in  Purkinje  cells 
from  young  kittens.  Such  a  structure  must  either  function 
differently  than  other  synapses  or  give  the  neuron  negative 
feedback.  In  the  latter  case*  the  autapse  would  be  inhibitory  and 
therefore  this  mechanism  would  only  be  expected  on  inhibitory 
neurons. 

The  function  f(x)  =  27  ln(x+l)  where  x  is  the  total  membrane 

current  in  jiamp/cm  has  been  computed  by  Agin  (1964)  from  the 

equations  of  Hodgkin  and  Huxley  (1952)  to  give  the  response 

frequency  (pulses/sec)  of  an  axon.  Strictly  speaking*  this  function 

is  not  bounded*  however  it  increases  very  slowly  for  large  x  and 

behaves  like  a  saturating  function  in  other  respects.  Keep  in  mind 

that  x  is  bounded  in  practice.  Note  that  by  (52)  this  function 

» 

satisfies  the  condition  (43)  and  hence  by  Theorem  4  it  gives  a  ♦  with 
only  one  nontrivial  aero. 


<x  -  In ( 1+x) )  >  0 


x 

m  J 

(1+x) 

for  x  >  0 

The  equation  for  the  system  in  Figure  30  can  be  rewritten  in  the 
N+l  dimensional  form  of  (53).  A  formalism  inspired  by  the  four- 
vector  approach  to  relativity  theory  in  physics  can  now  be  presented 
for  the  D-cell. 


(53) 


Consider  a  linear  range  for  such  that  02(x)  =  x.  Then 

the  (N+l) -vector 8  (m,q)  and  (d»°j(x))  can  be  considered  to  consist 
of  N  components  describing  the  spatial  pattern  of  afferent  activity 
and  connectivity  to  the  neuron  and  a  single  temporally  varying 
spatial  integration  and  self-connectivity.  The  metric  g  (54)  allows 
♦  (=  x  -  qo^(x))  to  be  written  as  the  inner  product  of  (m,q)  and 
(d,Oj(x))  giving  (55). 
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M  -  (M*D)D 

where  M  =  (m,q)  D  5  (d,o^) 


(55) 


Note  further  that  all  components  of  M  are  'synaptic'  and  those  of  D 
are  measures  of  axonal  activities.  The  postsynaptic  activity  x  is 
then  just  the  N-product  of  m  and  d  while  the  modulatory  signal  ♦  is 
the  (N+l)  'scalar'  M*D. 

Atttagottia.tifc  MechaaianB 

The  simplicity  of  the  approach  in  Part  III*  namely  that  two 
mechanisms  work  against  one  another  to  give  the  modulatory  function 
♦,  is  important.  Together  with  the  parallel  aspect  of  modification 
(nigd).  this  permits  the  system  equation  to  be  separated  into  a  few 
straightforward  principles  of  synaptic  modification: 

1.  The  efficacy  of  an  afferent's  signal  to  a  given 
target  neuron  varies  over  time  in  proportion  to  that 
(instantaneous)  signal.  (Parallel  modification) 

2.  Two  mechanisms*  which  are  dependent  on  the 
postsynaptic  activity*  determine  the  ratio  of  efficacy 
change  to  input  signal.  One  strengthens  the  synapses; 
the  other  weakens  them. 


3.  The  mechanisms  are  coupled  by  a  variable  quantity 
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which  varies  linearly  vitb  the  postsynaptic  activity 

accord ing  Jto.  Ike.  rnt  ratio. 

Aa  long  aa  the  functional  dependence  of  each  mechanism  on  the 
cell's  activity  aatiafiea  (37)*  the  neuron  will  aeek  either  minimum 
or  maximum  selectivity*  depending  on  the  sign  of  the  Wronkskian 
W( 0} * <>2 )  alone.  In  terms  of  the  earlier  presentations  of  the 
model  in  Part  II*  the  sign  of  determines  whether  the 

monotone  function  x  =  <Kq)  is  increasing  or  decreasing.  Thus  it  is 
observed  that  the  feature-abstracting  character  of  a  neuron  can  be 
determined  either  by  the  relative  signs  of  a  and  q  (the  parameter 
°)  or  the  mono tonic  trend  of  0(q). 


&*.  Cognitive  Application! 

Four  neuronal  types  can  be  straightforwardly  defined  based  on 
the  models  described  here.  The  afferent s  of  each  neuron  can  be 
tuned  for  either  minimum  (S-cell)  or  maximum  (D-cell)  selectivity 
over  a  single  stimulus  parameter  and  the  efferent  action  is  either 
excitatory  or  inhibitory.  It  is  hypothesised  that  a  network 
consisting  exclusively  of  these  four  cell  types  can  be  designed  to 
perform  arbitrarily  complex  similarity-difference  perceptual  tasks. 
That  is*  such  a  network  should  be  able  to  construct  a  taxonomy 
appropriate  to  any  stimulus  environment.  Certainly  more  easily  said 
than  done*  this  hypothesis  is  nevertheless  difficult  to  disprove.  It 


is  offered  es  •  challenge  to  provoke  the  realisation  of  the  full 
potential  of  such  circuits.  In  this  subsection  sow  initial  ateps 
to  test  this  hypothesis  are  omalined. 

It  nust  be  emphasised  from  the  outset  that  this  discussion  is 
limited  to  the  cognitive  phenomenon  of  hierarchical  abstraction,  i.e. 
classification  of  stimuli  by  their  similarities  and  differences.  The 
activity  of  each  neuron  simultaneously  modulates  the  modification  of 
all  its  input  synapses  in  proportion  to  the  activities  of  their 
individual  efferents  relative  to  an  internal  time- varying  scalar 
parameter.  This  mechanism  allows  individual  units  to  evolve  such  that 
they  abstract  simple  (scalar)  information  from  their  stimulus 
environments.  In  this  sense,  the  above  hypothesis  and  the 
mathematical  model  given  in  this  thesis  are  consistent  with  the 
single-unit  approach  of  Barlow  (1972).  His  five  dogmas  rely  on  the 
notion  of  a  neuron's  'trigger  feature'  —  that  aspect  of  a  stimulus 
which  excites  the  neuron.  In  the  picture  of  the  nervous  system 
presented  here*  stimuli  drive  neuronal  states  to  becow  maximally 
sensitive  to  certain  trigger  features.  These  My  be  specific 
'redundant  patterns  of  stimulation'  (D-  cells)  as  in  Barlow's  third 
dogma  or  generalised  prototypical  aspects  common  to  various  stimulus 
patterns  that  are  rarely  or  never  observed  as  pure  inputs  (8-cells). 

As  is  co—only  stated  (implicitly  or  explicitly),  the  nervous 
system  is  thought  to  be  organised  in  stages,  progressing  in 
perceptual  complexity  from  sensation  to  recognition,  association,  and 


cognition.  So.  it  is  proposed,  should  the  hypothetical  feature 
abstracting  network  be  organised.  A  genetically  coded  macrostructure 
lends  efficacy  and  relevance  ti  environmentally  influenced  learning 
at  the  cellular  level  in  that  whereas  each  neuron  is  modifiable  and 
adapts  to  informational  qualities  in  its  environment,  the  overall 
design  of  the  system  is  genetically  programmed  so  that  increasingly 
complex  information  is  abstracted  by  successive  stages  of  neural 
processing. 

Shepard  and  Podgorny  (1975)  point  out  that  symbolic  stimuli  are 
discretely  coded  while  physical  (nonsymbolie)  stimuli  vary 
continuously  along  one  or  several  dimensions:  "For.  whereas  we  can 
continuously  shift  a  color  (for  example,  blue)  in  brightness,  hue. 
and  saturation  until  it  becomes  as  similar  as  we  wish  to  any  other 
color  (for  example,  green),  we  cannot  continuously  deform  a  word 
'blue'  into  another  word  'green*  without  passing  through  intermediate 
configurations  that  are  not  words  at  all**.  Thus  it  is  expected  that 
inputs  to  the  higher  level  (more  "gnostic**)  stages  are  more 
distinctly  separated  than  early  (sensory)  input  patterns.  Two  candi¬ 
dates  for  implementing  such  separation  between  inputs  arc  strategic 
coding  and  tonic  inhibition.  Vectors  of  very  high  dimensionality 
tend  to  be  nearly  orthogonal  and  are  hence  intrinsically  well- 
separated.  Tonic  inhibition  can  enhance  separation  by  quashing 
responses  to  mediocre  excitation. 

The  perception  of  differences  and  similar ites  by  human  subjects 
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f 


hai  been  studied  by  several  researchers  (e.g.  Bosch*  1975*  Tver sky 
and  Gati*  1978*  1982).  The  considerable  mount  of  data  these  studies 
have  generated  ought  to  provide  tests  for  theoretical  hierarchy- 
abstracting  networks.  For  exsap le.  the  response  to  stiaulus  pattern 
A  of  a  neuron  broadly  tuned  to  stiaulus  pattern  B  My  measure  the 
perceived  sinilarity  of  A  to  B.  Such  a  definition  is  consistent 
with  Tver sky’*  (1977)  notion  of  asynaetry  judgements  expressed  by 
a  contrast  aodel  (56).  Such  aatheaatical  formulations  provide  a 
aeans  of  comparing  the  performances  of  aodel  networks  with  behavioral 
data. 


D(i.j)  -  of(i-J)  +  Sf(j-i)  -  ef(iHj) 

0,0,0  >  0 


(56) 


As  a  final  point*  the  value  of  this  approach  to  understanding 
the  relationships  between  neural  and  cognitive  phenomena  is 
considered.  The  complex  information-processing  capabilities  of 
biological  networks  can  only  be  understood  via  the  increased 
application  of  sophisticated  Mtheaatical  tools  to  neuroscience. 
8uch  efforts  My  provide  brain  research  with  the  long-awaited 
unravelling  of  the  neural  Mchanisas  underlying  learning  and 
cognition.  The  Mpping  of  a  MtheMtical  structure  for  Mural 
network  processing  onto  a  Mtheaatical  aodel  of  cognitive 
perforMoce  is  likely  to  be  the  unifying  event  in  this  undertaking. 
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Ha.  The  S«np«i  gwaawhla  Method 

Date  from  a  variety  of  investigative  nodes  (linguistic* 
psychological*  electrophysiological*  and  neurophantacho logical)  have 
been  subjected  to  analytical  treatment  of  an  increasingly  sophis¬ 
ticated  mathematical  nature  over  the  last  few  decades.  As  synaptic 
modification  models  evolve*  a  growing  interaction  between  model 
development  and  laboratory  research  is  anticipated.  The  method  for 
electrophysiological  testing  put  forward  in  Part  IV  is  suited  to  the 
current  state  of  laboratory  technique.  It  promises  to  narrow  the 
allowed  classes  of  models  for  certain  synaptic  networks  by  revealing 
aspects  of  their  plasticity  properties  near  equilibrium  states. 

The  notion  of  swasuring  a  synapse  ensemble  efficacy  (SEE)  is  an 
important  one.  It  is  based  on  the  following  assumption:  Aa  the  net 
reaPQBJB.e  fit  A  neural  pogulatifln  Ifi  A  constant  atimilua  pattern  from  a 
projecting  population  ox.  populations  approaches  aa  asymptotic  value, 
the  activity  oi  each  individual  neuron  converges  to.  a  limit  and 
iuxtheraore*  an  does  the  efficacy  ol  each  axnanae*  This  should  be 
considered  as  nothing  more  than  a  working  hypothesis  until  it  is 
either  proven  false  or  no  longer  necessary  as  single-cell  methods 
improve.  The  notions  of  convergence  and  asymptotic  approach  must  be 
carefully  considered  by  the  experimenter*  for  they  have  a  strong 
is^act  on  data  interpretation. 
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The  tent  “active"  and  “passive"  Modification  respectively 
correspond  to  the  notions  of  supervised  and  unsupervised  learning 
used  in  theories  of  knowledge  acquisition  (see  Cooper*  1973)*  Both 
types  require  that  the  organism  (student)  abstract  information  from  a 
stimulus  environment  (the  world).  Passive  learning  is  a  fully  aolf- 
orsaniaine  process  while  active  learning  requires  a  teacher  or 
supervisor  to  rectify  incorrect  conclusions  drawn  by  the  inductive 
process*  thereby  directing  and  accelerating  the  organisation  of  the 
system.  Analogously*  individual  neurons  or  neuron  populations  are 
students  undergoing  a  process  of  synaptic  modification  so  that  they 
“learn"  to  respond  appropriately  to  their  stimulus  environment.  If  a 
projection  has  sufficient  strength  to  control  a  population  it  can 
perform  the  teacher  role  in  associatively  modulating  the  plasticity 
of  weaker  projections.  However  the  strong  projection  must  modify 
passively  unless  it  receives  very  weak  stimulation  while  another 
(convergent)  bundle  “teaches". 


APBCPrfi*  4*  Ihg  Modal  Heuron  as  a  Signal  Dtfetor 


In  multipattern  stimulus  environments*  O-cells  seek  m*«imusi 
selectivity  end  S-cells  seek  minimum  selectivity.  In  the  context  of 
e  one-pet tern  environment*  selectivity  is  no  longer  well-defined. 

The  perellel  modification  models  (both  S  and  D)  described  in  this 
thesis  drive  the  synaptic  state  parallel  to  the  pattern  until  (Al) 
ia  satisfied  (Figure  31a). 

m*d  ■  B(q)  (Al) 


The  addition  of  a  decay  term  (antiparallel  to  m)  to  the  function 
driving  synaptic  modification  causes  the  components  of  m  not 
parallel  to  d  to  vanish  (Figure  31b) »  thereby  forcing  m  to  a  final 
state  parallel  to  d.  The  decay  term  must  be  carefully  chosen  to 
ensure  that  not  m  does  not  vanish  completely.  Consider  a  linear 
decay  (A2). 

m  ■  ♦(x.q)d  -  cm  (A2) 


This  works  for  siq  sufficiently  large*  however  the  state  m  s  0  is 
unstable.  By  using  a  decay  term  that  goes  as  a  higher  power  of  m* 
this  problem  is  solved  (A3). 

m  ■  ♦(x,q)d  “  *(■!•  (a$) 


Randcm  or  noisy  components  to  the  signal  cause  perturb  the 
state*  but  if  each  aodificattoit  to  m  is  not  too  large  (i.e.  for 
time  steps  in  the  difference  equation)*  the  state  remains  nearly 
parallel  to  d  forever  (Figure  31c). 


Figure  1L*.  Extraction  q£  a  signal  froa  noiae.  A  single  pattern  is 
presented  to  a  D-cell  in  this  2-diaensional  siauliation.  (a)  no 
decay.  no  noise,  (b)  linear  decay*  no  noise,  (c)  linear  decay 
with  noise. 


So  for  8  <  ♦m,  the  reel  parts  of  both  roots  are  positive  and 
therefore  a  liait  cycle  exists.  For  6  >  the  trajectories 

converge  to  the  critical  point.  Hote  that  for  Figure  8,  ^  s  J  s 
0.785  vas  used. 
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Appendix  j^.  Stability  fi£  the 


P-cell  about  ths.  Critical  faint  in usd  £  (1»1) 


The  behavior  of  the  systen  (14)  is  analysed  about  the  critical 
point  (1*1)  in  this  appendix.  From  Theorem  1  it  is  understood  that 
if  trajectories  diverge  locally*  they  must  eventually  converge  to  a 
limit  cycle  on  a  more  global  scale.  Consider  the  perturbation 
(m.q)  =  (l+x,l+y).  This  gives  a  linear  first  order  system  (Bl)  for  x 
and  y. 


x  -  *  (mpl,q-l)  [x  -  2y] 

n 

y  -  6  [x  -  y] 


The  solutions  of  (Bl)  are  of  the  form  (B2): 


(y)  ’  (**)exp<rit)  + 


vhere  r^  and  ^  (both  non>zero)  satisfy  (B3): 


therefore 


r  +  (3  -  *  (1,1)) r  +  ♦  B  *  0 
n  in 


r  -  +6v} 


Ag&Cfldia  £•  Computer  Simulation  cl  AH  Sr C, ell  Inhibiting  a  D-Cell 


The  circuit  used  in  this  simulation  is  described  in  Part  Va 
(Figure  27). 


ENVIRONMENT  (10  patterns  in  7  dimensions): 


pattern 

components 

1.518 

0.076 

0.000 

1.722 

0.000 

0.076 

1.518 

2 

1.677 

0.000 

0.016 

1.729 

0.016 

0.000 

1.677 

3 

0.917 

0.000 

0.417 

1.713 

0.417 

0.000 

0.917 

4 

0.224 

0.000 

1.235 

1.717 

1.235 

0.000 

0.224 

5 

0.000 

0.000 

1.732 

1.732 

1.732 

0.000 

0.000 

6 

0.000 

0.224 

1.235 

1.717 

1.235 

0.224 

0.000 

7 

0.000 

0.917 

0.417 

1.713 

0.417 

0.917 

0.000 

8 

0.000 

1.677 

0.016 

1.729 

0.016 

1.677 

0.000 

9 

0.076 

1.518 

0.000 

1.722 

0.000 

1.518 

0.076 

10 

0.644 

0.644 

0.000 

1.711 

0.000 

0.644 

0.644 

Initial 

state 

of 

S  - 

cell 

:  1  0  0  1 

0 

0 

1 

Initial 

state 

of 

D  - 

cell 

:  1  1  1  1 

1 

1 

1 

Initial 

value 

of 

k  : 

0 

pattern 

1 

2 

3 

4 

5 

6 

7 

8  9 

10 

t  =  0 

k  = 

0.00 

q(D-cell)  = 

0.00 

q(S-cell) 

=  0.00 

D-cell: 

4.91 

5.11 

4.38 

4.63 

5.20 

4.63  4.38 

5.11  4.91 

4.29 

S-cell: 

4.76 

5.08 

3.55 

2.16 

1.73 

1.72  1.71 

1.73  1.87 

3.00 

t  =  100 

k  = 

1.48 

q(D-cell)  = 

0.99 

q(S-cell) 

=  0.99 

D-cell: 

1.34 

1.53 

0.67 

0.04 

0.00 

0.00  0.00 

0.00  0.00 

0.39 

S-cell: 

1.06 

1.10 

0.95 

0.96 

1.04 

0.94  0.90 

1.03  0.99 

0.92 

O 

o 

CM 

II 

k  = 

1.11 

q(D-cell)  = 

2.47 

q(S-cell) 

a  0.99 

D-cell: 

7.77 

8.57 

4.69 

0.98 

0.00 

0.00  0.00 

0.00  0.16 

3.26 

S-cell: 

1.02 

1.05 

0.93 

0.97 

1.07 

0.97 

0.93 

1.05  1.02 

0.91 

t  =  300 

k  = 

1.30 

q(D-cell)  = 

2.90 

q(S-cell) 

=  0.99 

D-cell: 

9.63 

10.73 

5.41 

0.36 

0.00 

0.00  0.00 

0.00  0.00 

3.46 

S-cell: 

1.02 

1.05 

0.93 

0.97 

1.06 

0.97 

0.93 

1.05  1.02 

0.92 
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t  =  400  k  = 

1.77 

q(D-cell)  = 

3.15 

q(S-cell)  = 

«  0.99 

D-cell:  10.47 

11.69 

5.76  0.09 

0.00 

0.00  0.00 

0.00 

0.00 

3.58 

S-cell:  1.02 

1.05 

0.93  0.97 

1.06 

0.97  0.93 

1.05 

1.02 

0.92 

t  =  500  k  = 

2.25 

q(D-cell)  = 

3.23 

q(S-cell)  = 

:  0.99 

D-cell:  10.83 

12.13 

5.82  0.00 

0.00 

0.00  0.00 

0.00 

0.00 

3.49 

S-cell:  1.02 

1.05 

0.94  0.97 

1.06 

0.97  0.94 

1.05 

1.02 

0.92 

t  =  600  k  = 

2.75 

q(D-cell)  = 

3.27 

q(S-cell)  = 

:  0.99 

D-cell:  11.08 

12.45 

5.81  0.00 

0.00 

0.00  0.00 

0.00 

0.00 

3.35 

S-cell:  1.02 

1.05 

*.94  0.98 

1.06 

0.98  0.94 

1.05 

1.02 

0.92 

t  =  700  k  = 

3.25 

q(D-cell)  = 

3.31 

q(S-cell)  - 

:  0.99 

D-cell:  11.33 

12.77 

5.79  0.00 

0.00 

0.00  0.00 

0.00 

0.00 

3.20 

S-cell:  1.02 

1.05 

0.94  0.98 

1.06 

0.98  0.94 

1.05 

1.02 

0.93 

t  =  800  k  = 

3.76 

q(D-cell)  = 

3.35 

qCS-cell)  = 

:  0.99 

D-cell:  11.59 

13.09 

5.77  0.00 

0.00 

0.00  0.00 

0.00 

0.00 

3.05 

S-cell:  1.01 

1.04 

0.94  0.98 

1.06 

0.98  0.94 

1.04 

1.01 

0.93 

t  =  900  k  = 

4.26 

q(D-cell)  = 

3.39 

q(S-cell)  = 

0.99 

D-cell:  11.84 

13.41 

5.74  0.00 

0.00 

0.00  0.00 

0.00 

0.00 

2.89 

S-cell:  1.01 

1.04 

0.94  0.98 

1.05 

0.98  0.94 

1.04 

1.01 

0.93 

t  =  1000  k  = 

«  4.77 

q(D-cell)  : 

=  3.43  q(S-cell) 

=  0. 

99 

D-cell:  12.10 

13.74 

5.72  0.00 

0.00 

0.00  0.00 

0.00 

0.00 

2.73 

S-cell:  1.01 

1.04 

0.95  0.98 

1.05 

0.98  0.95 

1.04 

1.01 

0.93 

t  =  2000  k  « 

:  9.76 

q(D-cell)  * 

3.81 

q(S-cell)  = 

1.00 

D-cell:  14.63 

17.00 

5.42  0.00 

0.00 

0.00  0.00 

0.00 

0.00 

1.11 

S-celi:  1.01 

1.03 

0.96  0.98 

1.04 

0.98  0.96 

1.03 

1.01 

0.95 

t  =  3000  k  = 

:  13.38  q(D-cell) 

=  4. 

12  q(S-cell) 

=  1 

00 

D-cell:  16,55 

19.49 

5.14  0.00 

0.00 

0.00  0.00 

0.00 

C.<K> 

S-cell:  1.01 

1.02 

0.97  0.99 

1.03 

0.99  0.97 

1.02 

1.01 

0.97 

t  =  4000  k  = 

=  16.32  q(D-cell) 

=  4. 

20  q(S-cell) 

=  1 

.00 

D-cell:  17.20 

20.53 

4.32  0.00 

0.00 

0.00  0.00 

0.00 

0.00 

0.00 

S-cell:  1.01 

1.02 

0.98  0.99 

1.02 

0.99  0.98 

1.02 

1.01 

0.97 

t  =  5000  k  = 

■  19.28  q(D-celi) 

=  4. 

29  q(S-cell) 

=  1 

.00 

D-cell:  17.85 

21.58 

3.48  0.00 

0.00 

0.00  0.00 

0.00 

0.00 

0.00 

S-cell:  1.00 

1.01 

0.99  0.99 

1.02 

0.99  0.99 

1.01 

1.00 

0.98 

t  =  6000  k  = 

:  22.20  q(D-celi) 

=  4. 

38  q(S-cell) 

=  1 

.00 

D-cell:  18.50 

22.62 

2.66  0.00 

0.00 

0.00  0.00 

0.00 

0.00 

0.00 

S-cell:  1.00 

1.01 

0.99  1.00 

1.01 

1.00  0.99 

1.01 

1.00 

0.99 

t  =  7000  k  = 

*  24.98 

t  q(D-cell) 

=  4.' 

47  q(S-cell) 

=  1 

.00 

D-cell:  19.15 

23 .64 

1.89  0.00 

0.00 

0.00  0.00 

0.00 

0.00 

0.00 

S-celi:  1.00 

1.01 

0.99  1.00 

1.01 

1.00  0.99 

1.01 

1.00 

0.99 

t  =  8000  k  =  27.48  q(D-cell)  =  4.57  q(S-cell)  =  1.00 

D-cell:  19.80  24.64  1.25  0.00  0.00  0.00  0.00  0.00  0.00  0.00 

S-cell:  1.00  1.01  0.99  1.00  1.01  1.00  0.99  1.01  1.00  0.99 

t  =  9000  k  =  29.58  q(D-cell)  =  4.68  q(S-cell)  =  1.00 

D-cell:  20.44  25.56  0.78  0.00  0.00  0.00  0.00  0.00  0.00  0.00 

S-cell:  1.00  1.00  1.00  1.00  1.01  1.00  1.00  1.00  1.00  0.99 

t  =  10000  k  =  31.25  q(D-cell)  =  4.79  q(S-cell)  =  1.00 

D-cell:  21.03  26.38  0.49  0.00  0.00  0.00  0.00  0.00  0.00  0.00 

S-cell:  1.00  1.00  1.00  1.00  1.01  1.00  1.00  1.00  1.00  0.99 

t  =  20000  k  =  38.65  q(D-cell)  =  5.12  q(S-cell)  =  1.00 

D-cell:  22.44  28.71  0.00  0.00  0.00  0.00  0.00  0.00  0.00  0.00 

S-cell:  1.00  1.00  1.00  1.00  1.01  1.00  1.00  1.00  1.00  0.99 

t  =  30000  k  =  44.92  q(D-cell)  =  5.14  q(S-cell)  =  1.00 

D-cell:  22.24  29.16  0.00  0.00  0.00  0.00  0.00  0.00  0.00  0.00 

S-cell:  1.00  1.00  1.00  1.00  1.01  1.00  1.00  1.00  1.00  0.99 

t  =  40000  k  =  51.30  q(D-cell)  =  5.17  q(S-cell)  =  1.00 

D-cell:  22.05  29.63  0.00  0.00  0.00  0.00  0.00  0.00  0.00  0.00 

S-cell:  1.00  1.00  1.00  1.00  1.01  1.00  1.00  1.00  1.00  0.99 

t  =  50000  k  =  57.78  q(D-cell)  =  5.20  q(S-cell)  =  1.00 

D-cell:  21.87  30.12  0.00  0.00  0.00  0.00  0.00  0.00  0.00  0.00 

S-cell:  1.00  1.00  1.00  1.00  1.01  1.00  1.00  1.00  1.00  0.99 

t  =  100000  k  =  91.23  q(D-cell)  =  5.38  q(S-cell)  *  1.00 

D-cell:  21.04  32.78  0.00  0.00  0.00  0.00  0.00  0.00  0.00  0.00 

S-cell:  1.00  1.00  1.00  1.00  1.01  1.00  1.00  1.00  1.00  0.99 

t  =  150000  k  =  125.77  q(D-cell)  *  5.60  q(S-cell)  =  1.00 

D-cell:  20.32  35.68  0.00  0.00  0.00  0.00  0.00  0.00  0.00  0.00 

S-cell:  1.00  1.00  1.00  1.00  1.01  1.00  1.00  1.00  1.00  0.99 

t  =  200000  k  =  160.84  q(D-cell)  =  5.84  q(S-cell)  *  1.00 

D-cell:  19.66  38.70  0.00  0.00  0.00  0.00  0.00  0.00  0.00  0.00 

S-cell:  1.00  1.00  1.00  1.00  1.01  1.00  1.00  1.00  1.00  0.99 

t  =  250000  k  =  196.27  q(D-cell)  =  6.07  q(S-cell)  =  1.00 

D-cell:  18.98  41.75  0.00  0.00  0.00  0.00  0.00  0.00  0.00  0.00 

S-cell:  1.00  1.00  1.00  1.00  1.01  1.00  1.00  1.00  1.00  0.99 

t  =  300000  k  =  232.13  q(D-cell)  =  6.31  q(S-cell)  *  1.00 

D-cell:  18.29  44.81  0.00  0.00  0.00  0.00  0.00  0.00  0.00  0.00 

S-cell:  1.00  1.00  1.00  1.00  1.01  1.00  1.00  1.00  1.00  0.99 

t  =  350000  k  =  267.99  q(D-cell)  =  6.54  q(S-cell)  =  1.00 

D-cell:  17.56  47.84  0.00  0.00  0.00  0.00  0.00  0.00  0.00  0.00 

S-cell:  1.00  1.00  1.00  1.00  1.01  1.00  1.00  1.00  1.00  0.99 
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t  =  400000  k 
D-cell:  16.78 
S-cell:  1.00 

t  =  450000  k 
D-cell s  15.97 
S-cell:  1.00 

C  =  500000  k 
D-cell:  15.11 
S-cell:  1.00 

t  =  550000  k 
D-cell:  14.25 
S-cell:  1.00 

t  =  600000  k 
D-cell:  13.37 
S-cell:  1.00 

t  =  650000  k 
D-cell:  12.46 
S-cell:  1.00 

t  =  700000  k 
D-cell:  11.51 
S-cell:  1.00 

t  a  750000  k 
D-cell:  10.53 
S-cell:  1.00 

t  =  800000  k 
D-cell:  9.54 
S-cell:  1.00 

t  =  900000  k 
D-cell:  7.60 
S-cell:  1.00 

t  a  1000000  I 
D-cell:  5.71 
S-cell:  1.00 

t  »  1100000  1 
D-cell:  3.83 
S-cell:  1.00 

t  a  1200000  1 

D-cell:  2.18 
S-cell:  1.00 


a  304. 

61 

q(D-cell) 

1  =  6 

.77 

q(S-cell) 

3 

1.00 

50.89 

0 

.00 

0.00 

0.00 

0.00 

0.00 

0 

.00 

0.00 

0.00 

1.00 

1 

.00 

1.00 

1.01 

1.00 

1.00 

1 

.00 

1.00 

0.99 

=  341. 

,23 

s' 

(D-cell] 

1  a  6 

.99 

q(S-cell) 

3 

1.00 

53.89 

0 

.00 

0.00 

0.00 

0.00 

0.00 

0 

.00 

0.00 

0.00 

1.00 

1 

.00 

1.00 

1.01 

1.00 

1.00 

1 

.00 

1.00 

0.99, 

=  377. 

72 

q(D-cell) 

'  a  7 

.20 

q(S-cell) 

s 

1.00 

56.85 

0 

.00 

0.00 

0.00 

0.00 

0.00 

0 

.00 

0.00 

0.00 

1.00 

1 

.00 

1.00 

1.01 

1.00 

1.00 

1 

.00 

1.00 

0.99 

a  413. 

29 

q(D-cell) 

a  7 

.40 

q(S-cell) 

ss 

1.00 

59.70 

0 

.00 

0.00 

0.00 

0.00 

0.00 

0 

.00 

0.00 

0.00 

1.00 

1 

.00 

1.00 

1.01 

1.00 

1.00 

1 

.00 

1.00 

0.99 

=  448. 

89 

q(D-cell) 

a  7 

.59 

q(S-cell) 

1.00 

62.53 

0 

.00 

0.00 

0.00 

0.00 

0.00 

0 

.00 

0.00 

0.00 

1.00 

1 

.00 

1.00 

1.01 

1.00 

1.00 

1 

.00 

1.00 

0.99 

=  484. 

52 

q(D-cell) 

a  7 

.78 

q(S-cell) 

s 

1.00 

65.34 

0 

.00 

0.00 

0.00 

0.00 

0.00 

0 

.00 

0.00 

0.00 

1.00 

1 

.00 

1.00 

1.01 

1.00 

1.00 

1 

.00 

1.00 

0.99 

a  520. 

39 

q(D-cell) 

a  7 

.96 

q(S-cell) 

S 

1.00 

68.12 

0 

.00 

0.00 

0.00 

0.00 

0.00 

0 

.00 

0.00 

0.00 

1.00 

1 

.00 

1.00 

1.01 

1.00 

1.00 

1 

.00 

1.00 

0.99 

=  557. 

01 

q(D-cell) 

a  8 

.15 

q(S-cell) 

s 

1.00 

70.95 

0 

.00 

0.00 

0.00 

0.00 

0.00 

0 

.00 

0.00 

0.00 

1.00 

1 

.00 

1.00 

1.01 

1.00 

1.00 

1 

.00 

1.00 

0.99 

=  593. 

40 

q(D-cell) 

a  8 

.33  ( 

q(S-cell) 

s 

1.00 

73.74 

0 

.00 

0.00 

0.00 

0.00 

0.00 

0 

.00 

0.00 

0.00 

1.00 

1 

.00 

1.00 

1.01 

1.00 

1.00 

1 

.00 

1.00 

0.99 

=  663. 

15 

q(D-cell) 

=  8 

.67  1 

q(S-cell) 

s 

1.00 

79.07 

0 

.00 

0.00 

0.00 

0.00 

0.00 

0 

.00 

0.00 

0.00 

1.00 

1 

.00 

1.00 

1.01 

1.00 

1.00 

1 

.00 

1.00 

0.99 

t  a  730 

.29  q(D-cell)  =  1 

3.99 

q(S-cell)  * 

1.00 

84.17 

0 

.00 

0.00 

0.00 

0.00 

0.00 

0 

.00 

0.00 

0.00 

1.00 

1. 

.00 

1.00 

1.01 

1.00 

1.00 

1 

.00 

1.00 

0.99 

l  =  797 

.08  q(D-cell)  a  9 

.30  • 

i(S-cell) 

S 

1.00 

89.22 

0, 

.00 

0.00 

0.00 

0.00 

0.00 

0, 

.00 

0.00 

0.00 

1.00 

1, 

.00 

1.00 

1.01 

1.00 

1.00 

1, 

.00 

1.00 

0.99 

L  a  856 

.50  q(D-cell)  a  9, 

.59  q(S-celi) 

s 

1.00 

93.75 

0, 

.00 

0.00 

0.00 

0.00 

0.00 

0, 

.00 

0.00 

0.00 

1.00 

1, 

.00 

1.00 

1.01 

1.00 

1.00 

1, 

.00 

1.00 

0.99 

85 


t  =  1300000  k  =  903 

.06 

q(D-cell)  «  9 

.82 

q(S-cell)  * 

1.00 

D-cell: 

0.91  97.33 

0.00 

0.00  0.00 

0.00 

0.00  0.00 

0.00 

0.00 

S-cell: 

1.00  1.00 

1.00 

1.00  1.01 

1.00 

1.00  1.00 

1.00 

0.99 

t  =  1400000  k  =  926 

.71 

q(D-cell)  =  9 

.95 

q(S-ceii)  * 

1.00 

D-cell: 

0.28  99.17 

0.00 

0.00  0.00 

0.00 

0.00  0.00 

0.00 

0.00 

S-cell: 

1.00  1.00 

1.00 

1.00  1.01 

1.00 

1.00  1.00 

1.00 

0.99 

t  =  1500000  k  =  934.69 

q(D-cell)  =  9 

.99 

q(S-cell)  * 

1.00 

D-cell: 

0.07  99.79 

0.00 

0.00  0.00 

0.00 

0.00  0.00 

0.00 

0.00 

S-cell: 

1.00  1.00 

1.00 

1.00  1.01 

1.00 

1.00  1.00 

1.00 

0.99 

t  =  1600000  k  =  935 

.10 

q(D-cell)  =  9 

.99 

q(S-cell)  = 

1.00 

D-cell: 

0.06  99.82 

0.00 

0.00  0.00 

0.00 

0.00  0.00 

0.00 

0.00 

S-cell: 

1.00  1.00 

1.00 

1.00  1.01 

1.00 

1.00  1.00 

1.00 

0.99 

t  *  1700000  k  =  935 

.10 

q(D-cell)  =  9 

.99 

q(S-cell)  * 

1.00 

D-cell: 

0.06  99.82 

0.00 

0.00  0.00 

0.00 

0.00  0.00 

0.00 

0.00 

S-cell: 

1.00  1.00 

1.00 

1.00  1.01 

1.00 

1.00  1.00 

1.00 

0.99 

t  =  1800000  k  =  935 

.10 

q(D-cell)  =  9 

.99 

q(S-cell)  = 

1.00 

D-cell: 

0.06  99.82 

0.00 

0.00  0.00 

0.00 

0.00  0.00 

0.00 

0.00 

S-cell: 

1.00  1.00 

1.00 

1.00  1.01 

1.00 

1.00  1.00 

1.00 

0.99 

t  =  1900000  k  =  935 

.10 

q(D-cell)  =  9 

.99 

q(S-cell)  * 

1.00 

D-cell: 

0.06  99.82 

0.00 

0.00  0.00 

0.00 

0.00  0.00 

0.00 

0.00 

S-cell: 

1.00  1.00 

1.00 

1.00  1.01 

1.00 

1.00  1.00 

1.00 

0.99 

t  =  2000000  k  =  935 

.10 

q(D-ceii)  =  9 

.99 

q(S-cell)  - 

1.00 

D-cell: 

0.06  99.82 

0.00 

0.00  0.00 

0.00 

0.00  0.00 

0.00 

0.00 

S-cell: 

1.00  1.00 

1.00 

1.00  1.01 

1.00 

1.00  1.00 

1.00 

0.99 
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